-- Logs begin at Mon 2022-05-09 14:24:42 UTC, end at Mon 2022-05-09 16:29:39 UTC. -- May 09 15:01:06.754862 dv-cinder-840678-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 09 15:01:06.762348 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 09 15:01:06.764300 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 9 15:01:06 2022] *** May 09 15:01:06.764300 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 09 15:01:06.764300 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 09 15:01:06.764300 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: nodename: dv-cinder-840678-3-iscsi May 09 15:01:06.764300 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: machine: x86_64 May 09 15:01:06.764300 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: clock source: unix May 09 15:01:06.764300 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: pcre jit disabled May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: detected number of CPU cores: 4 May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: current working directory: / May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: detected binary path: /usr/bin/uwsgi-core May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: your processes number limit is 47680 May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: your memory page size is 4096 bytes May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: detected max file descriptor number: 2048 May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: lock engine: pthread robust mutexes May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: thunder lock: enabled May 09 15:01:06.764792 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 09 15:01:06.765275 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 09 15:01:06.810611 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: Python main interpreter initialized at 0x55edb6c6c9c0 May 09 15:01:06.810611 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: python threads support enabled May 09 15:01:06.810611 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: your server socket listen backlog is limited to 100 connections May 09 15:01:06.810611 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: your mercy for graceful operations on workers is 90 seconds May 09 15:01:06.810956 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: mapped 403077 bytes (393 KB) for 2 cores May 09 15:01:06.811073 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: *** Operational MODE: preforking *** May 09 15:01:06.811350 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: *** uWSGI is running in multiple interpreter mode *** May 09 15:01:06.811350 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: spawned uWSGI master process (pid: 80049) May 09 15:01:06.812113 dv-cinder-840678-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 09 15:01:06.818211 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: spawned uWSGI worker 1 (pid: 80060, cores: 1) May 09 15:01:06.818211 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: spawned uWSGI worker 2 (pid: 80062, cores: 1) May 09 15:01:06.818211 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 09 15:01:12.589018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:01:12.589018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: from cryptography.utils import int_from_bytes May 09 15:01:12.589796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:01:12.589796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: from cryptography.utils import int_from_bytes May 09 15:01:12.590394 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:01:12.590394 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: from cryptography.utils import int_from_bytes May 09 15:01:12.590621 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:01:12.590621 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: from cryptography.utils import int_from_bytes May 09 15:01:12.640594 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 15:01:12.640848 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 15:01:12.640999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] command line args: [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 15:01:12.641188 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 15:01:12.641188 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 15:01:12.641320 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 15:01:12.641381 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] command line args: [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 15:01:12.641444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ================================================================================ {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 15:01:12.641619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 15:01:12.641707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] admin_token = **** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.641851 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ================================================================================ {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 15:01:12.641944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] config_dir = [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.642081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] admin_token = **** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.642165 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.642233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] config_source = [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.642291 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] config_dir = [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.642435 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] debug = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.642527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.642589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] config_source = [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] debug = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645750 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645750 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645924 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.645981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646044 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] list_limit = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] log_config_append = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646220 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] list_limit = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646280 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] log_config_append = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] log_dir = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] log_file = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646595 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] log_dir = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646595 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] log_file = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646703 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646888 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.646951 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647010 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647179 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647510 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647879 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647935 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.647998 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648121 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648504 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.648563 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.649419 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.649643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] publish_errors = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.649810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] publish_errors = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.650038 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.650252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.650419 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.650419 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.650785 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.650785 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.651153 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.651437 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.651672 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.651845 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.652039 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] standard_threads = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.652243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] standard_threads = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.652473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.652642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.652843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.653067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.653277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.653451 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.653683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] use_journal = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.653851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] use_journal = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.654050 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] use_json = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.654213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] use_json = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.654379 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] use_stderr = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.654537 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] use_stderr = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.654682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] use_syslog = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.654824 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] use_syslog = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.654987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.655781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:01:12.655917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.656047 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.656188 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.656325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.656462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.656598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.656735 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.656873 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.657004 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.657156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.657297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.657427 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.657556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.657765 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.657905 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.658037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.658172 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.658303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.658432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.658570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.658707 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.658837 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.658966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.659100 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.659241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.659383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.659521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.659784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.659936 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.660069 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.660205 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.660363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.660497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.660629 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.660759 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.660887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.661016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.661199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.661339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.661476 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.661607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.661738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.661883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.662022 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.662154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.662283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.662412 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.662540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.662668 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.662808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.662949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.663079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.663207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.663335 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.663463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.663600 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.663731 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.663861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.663995 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.664131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.664262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.664465 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.664604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.664735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.664881 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.665012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.665209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.665351 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.665482 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.665610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.665738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.665867 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.666008 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.666139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.666268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.666396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.666525 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.666663 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.666804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.666942 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.667073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.667202 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.667331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.667459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.667595 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.667725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.667854 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.667983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.668117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.668253 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.668391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.669660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.669841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.670036 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.670274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.670440 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.670673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.670840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.671035 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.671231 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.671425 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.671624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.671855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.672024 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.672305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.672529 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.672699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.672893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.673162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.673356 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.673550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.673745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.673970 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.674171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.674378 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.674547 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.674547 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.674859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.675003 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.675171 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.675310 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.675487 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.675623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.675764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.675901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.676041 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.676210 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.676346 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.676486 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.676619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.676748 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.676888 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.677018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.677226 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.677387 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.677552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.677686 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.677818 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.677955 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.678087 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.678225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.678363 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.678496 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.678626 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.678773 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.678906 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.679045 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.679177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.679313 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.679456 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.679590 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.679726 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.679866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.679998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.680168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.680332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.680464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.680595 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.680811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.680946 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.681120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.681293 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.681430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.681573 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.681707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.681838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.681968 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.682098 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.682228 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.682368 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.682501 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.682637 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.682780 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.682910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.683040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.683179 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.683318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.683509 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.683641 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.683772 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.683902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.684041 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.684181 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.684325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.684456 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.684587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.684728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.684860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.684991 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.685167 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.685314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.685446 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.685576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.685707 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.685838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.685976 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.686108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.686277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.686408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.686548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.686680 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.686819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.686951 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.687081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.687211 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.687340 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.687470 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.687611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.687749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] auth.external = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.687880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] auth.external = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.688011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.688141 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.688270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.688443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.688587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.688731 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.688871 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] auth.password = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.689005 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] auth.password = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.689183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] auth.token = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.689183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.689432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] auth.token = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.689574 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.689707 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.689838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.689969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.690101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.690260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.690399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.690530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.690676 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.690808 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.690939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.691079 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.691212 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] credential.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.691343 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.691473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.691604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] credential.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.691735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.691875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.692007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.692150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.692282 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.692412 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.692542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.692681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.692814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.692944 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.693100 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.693241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.693372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.693511 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.693652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.693785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.693925 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.694063 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.694196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.694335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.694469 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.694606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.694737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.694867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.695005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.695145 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.695279 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.695409 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.695540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] federation.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.695671 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.695811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.695955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] federation.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.696112 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.696347 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.696515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.696651 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.696783 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.696993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.697187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.697323 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.697461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.697594 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.697726 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.697881 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.698015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.698146 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.698277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.698410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.698541 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.698681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.698814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.698945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.699081 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.699212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.699342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.699485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.699617 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.699757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.699889 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.700021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.700152 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.700301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.700435 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.700566 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.700697 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.700828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.700957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.701137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.701278 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.701410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.701556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.701688 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.702324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.702463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.702595 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.702726 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.702857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.702988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.703127 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.703260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.703391 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.703522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.703653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.703784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.703933 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.704068 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.704199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.704331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.704462 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.704593 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.704733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.704867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.704998 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.705162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.705294 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.705424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.705566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.705566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.705827 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.706073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.706227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.706502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.706634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.706766 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.706897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.707027 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.707158 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.707297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.707430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.707561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.707702 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.707835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.707966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.708105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.708243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.708376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.708507 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.708637 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.708768 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.708907 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.709058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.709197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.709329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.709459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.709610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.709752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.709884 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.710015 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.710146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.710287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.710419 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.710559 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.710691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.710822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.710952 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.711084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.711213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.711363 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.711496 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.711635 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.711767 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.711898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.712029 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.712169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.712302 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.712433 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.712612 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.712751 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.712882 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.713021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.713212 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.713355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.713487 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.713617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.713747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.713887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.714019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.714152 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.714282 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.714412 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.714543 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.714681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.714814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.714945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.715076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.715217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.715348 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.715487 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.715620 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.715751 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.715882 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.716014 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.716153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.716286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.716417 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.716547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.716678 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.716809 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.716957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.717179 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.717316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.717447 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.718167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.718304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.718445 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.718578 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.718709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.718841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.718973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.719104 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.719243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.719377 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.719519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.719651 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.719782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.719912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.720052 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.720185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.720326 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.720459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.720590 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.720721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.720867 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.721002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.721189 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.721323 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.721466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.721598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.721740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.721873 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.722005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.722136 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.722267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.722398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.722539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.722672 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.722803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.722934 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.723064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.723205 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.723346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.723479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.723611 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.723741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.723872 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.724002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.724210 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] resource.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.724346 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.724478 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.724609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.724740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.724871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] resource.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.725010 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.725176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.725309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.725440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.725571 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.725702 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.725844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.725978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.726108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.726238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.726367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.726498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.726637 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.726770 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.726901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] role.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.727051 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.727184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] role.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.727314 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.727453 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.727586 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] role.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.727717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.727848 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] role.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.727979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.728110 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.728249 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.728382 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.728512 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.728643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.728786 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.728918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.729085 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.729226 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.729359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.729490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.729621 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.729752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.729895 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.730028 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.730159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.730296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.730430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.730570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.730711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.730844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.730975 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.731106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.731106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.731362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.731495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.731632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.731632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.731886 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.731886 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.732141 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.732273 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.732414 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.732546 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.732676 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.732807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.732946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.733119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.733744 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.733876 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.734006 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.734138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.734280 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.734280 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.734530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.734671 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.734802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.734933 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.735073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.735073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.735339 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.735476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.735615 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.735752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.735898 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.735898 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.736159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.736297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.736444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.736582 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.736728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.736868 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] token.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.737005 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.737194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.737333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.737482 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.737623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] token.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.737760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.737898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.738036 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.738173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.738173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.738455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.738455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.738715 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.738853 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.738990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.739136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.739276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.739413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.739413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.739673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.739810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.739956 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.740096 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.740234 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.740398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.740537 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.740675 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.740822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.740961 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.741120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.741262 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.741402 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.741402 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.741673 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.741815 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.741952 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.connection = **** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.742099 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.742258 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.742258 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.742523 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.742661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.742807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.connection = **** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.742955 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.743103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.743243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.743381 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.743521 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.743659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.743805 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.743945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.744083 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.744231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.744370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.744508 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.744656 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.744796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.744935 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.745090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.745234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.745374 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.745524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.745673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.745814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.745952 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.746090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.746239 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.746451 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.746595 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.746734 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.746872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.747012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.747151 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.747297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80062) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 15:01:12.747438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.747438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.747438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:01:12.747438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80060) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 15:01:12.757422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.757656 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.757880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.758295 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.758529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.758751 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.766650 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.767568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.796697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.797016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.797201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.797384 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.850131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.850806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.902234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.902544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.903383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.903622 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.973672 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.973914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.974129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.974350 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.974650 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.974920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.976297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.976535 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.976761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.976986 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.977227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.977444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.998522 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.998795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.998911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.999107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.999192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.999257 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.999462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.999552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.999627 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.999692 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:12.999926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.000010 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.000084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.000308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.011536 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.011742 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.026832 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.027061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.027309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.027562 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.027649 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.027819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.027903 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.027977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.028042 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.028290 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80062) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.028374 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.028438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.028636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.028842 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80060) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:01:13.046031 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.046241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:01:13.046326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.046495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:01:13.046592 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.046775 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:01:13.046856 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.047024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:01:13.047105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.047270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.047361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.047528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.047610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.047773 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.047853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.048029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.058772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.059044 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.059131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.059302 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.059384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.059567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.059648 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.059840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.068480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.072624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.072902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.073207 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.073608 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.075580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.075580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.075580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.075580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.075580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.075580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.075950 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.076018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.076081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.076081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.076605 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.076605 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.076747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.077591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.077864 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.078136 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.078410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.078677 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.102607 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.104174 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.104456 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.105439 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.107077 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.107358 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.109894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.110172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.110451 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.114532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.114532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.114532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.117869 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.118156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.122138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.122138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.122839 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.123124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.123400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.123677 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.124472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.124740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.125004 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.125293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.126787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.126787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.127105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.127311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.127583 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.127855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.128132 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.128399 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.144944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.145248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.148023 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.148300 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.149818 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.152841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.153520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.153814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.156522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.156817 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.158848 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.161836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.162499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.165449 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.166265 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.166560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.166840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.167129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.167403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.167691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.169207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.169508 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.169792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.170077 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.170355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.170693 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.187472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.187850 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.188119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.188384 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.190459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.190836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.191106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.191382 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.196303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.198069 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.198348 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.199327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.201066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.201345 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.201442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.201684 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.202539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.202817 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.203090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.204985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.205300 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.205582 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.205862 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.206138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.216597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.216876 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.217180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.217494 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.217773 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.218042 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.218321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.219726 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.220007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.220286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.220576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.220852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.221157 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.221433 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.236415 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.239711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.239894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.243055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.243218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.244639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.246294 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.246527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.246687 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.246856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.248135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.250335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.250615 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.250894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.254868 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.256730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.258676 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.259983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.260293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.260456 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.260597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.260809 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.263546 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.263840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.264123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.264979 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.270623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.270914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.271188 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.274268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.274567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.274846 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.279200 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80062) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.279478 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.279755 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.280053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.280323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.280597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.280878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.281176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.281444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.281711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.281972 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.282236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.282504 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.282758 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80062) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.283034 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80060) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:01:13.283311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.283590 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.284128 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.284410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.284690 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.284964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.285259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.285525 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.285790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.286063 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.286331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.286588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80060) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:01:13.321963 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55edb6c6c9c0 pid: 80062 (default app) May 09 15:01:13.325297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e39a02-ced0-4ed6-b4cc-ba3c83f405c9 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:13.325601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e39a02-ced0-4ed6-b4cc-ba3c83f405c9 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:13.325880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e39a02-ced0-4ed6-b4cc-ba3c83f405c9 None None] PATH_INFO: `/v3` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:13.326368 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55edb6c6c9c0 pid: 80060 (default app) May 09 15:01:13.330658 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon May 9 15:01:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:21.484860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36d03-2a98-4a74-8e6d-ec6ed2311416 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:21.485453 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36d03-2a98-4a74-8e6d-ec6ed2311416 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:21.485453 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c36d03-2a98-4a74-8e6d-ec6ed2311416 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:21.489687 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:21.495069 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:21.495535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:21.495626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:21.527379 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG oslo_db.sqlalchemy.engines [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80062) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 15:01:21.715506 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG passlib.handlers.bcrypt [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80062) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 15:01:21.715921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG passlib.handlers.bcrypt [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80062) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 15:01:21.771169 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:21.772700 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:21.893396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 09 15:01:22.085271 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c36d03-2a98-4a74-8e6d-ec6ed2311416 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:22.119944 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG oslo_db.sqlalchemy.engines [None req-55c36d03-2a98-4a74-8e6d-ec6ed2311416 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80060) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 15:01:22.248044 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c36d03-2a98-4a74-8e6d-ec6ed2311416 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:22.252739 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317a2cd-2af6-46de-95e3-20738f477622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3f51f62b70d42e281dc8a352a6f16d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:22.253418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317a2cd-2af6-46de-95e3-20738f477622 admin admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:22.253701 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317a2cd-2af6-46de-95e3-20738f477622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:22.253976 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317a2cd-2af6-46de-95e3-20738f477622 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:22.262135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317a2cd-2af6-46de-95e3-20738f477622 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:22.313316 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4317a2cd-2af6-46de-95e3-20738f477622 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 15:01:22.313652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317a2cd-2af6-46de-95e3-20738f477622 admin admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:22.323257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 9 15:01:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 15:01:22.329485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:22.344004 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216a1fe-d100-4f5f-9987-c8a52f46fc6c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:22.348304 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef932-8b7a-4e44-9656-27414c28bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3f51f62b70d42e281dc8a352a6f16d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:22.349433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef932-8b7a-4e44-9656-27414c28bfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:22.349717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef932-8b7a-4e44-9656-27414c28bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:22.349993 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef932-8b7a-4e44-9656-27414c28bfb8 admin admin] PATH_INFO: `/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:22.351519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ef932-8b7a-4e44-9656-27414c28bfb8 admin admin] RBAC: Authorizing `identity:get_project(project_id=f3f51f62b70d42e281dc8a352a6f16d6)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:22.402727 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: WARNING keystone.common.rbac_enforcer.enforcer [None req-736ef932-8b7a-4e44-9656-27414c28bfb8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 15:01:22.403073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ef932-8b7a-4e44-9656-27414c28bfb8 admin admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:22.408200 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon May 9 15:01:22 2022] GET /identity/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:24.615703 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80fd3-82bf-4444-90ee-871608ba12f9 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:24.616250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80fd3-82bf-4444-90ee-871608ba12f9 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:24.616250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b80fd3-82bf-4444-90ee-871608ba12f9 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:24.620391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:24.625887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c67c12-500b-48db-88a3-44c39e325ea3 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:24.626298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c67c12-500b-48db-88a3-44c39e325ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:24.626441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c67c12-500b-48db-88a3-44c39e325ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:24.646915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-27c67c12-500b-48db-88a3-44c39e325ea3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:24.648430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-27c67c12-500b-48db-88a3-44c39e325ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:24.656764 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 15:01:24.678710 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b80fd3-82bf-4444-90ee-871608ba12f9 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:24.690644 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b80fd3-82bf-4444-90ee-871608ba12f9 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:24.694842 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c30121-0c26-4aa7-acee-197689ed55c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3f51f62b70d42e281dc8a352a6f16d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:24.695471 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c30121-0c26-4aa7-acee-197689ed55c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:24.695755 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c30121-0c26-4aa7-acee-197689ed55c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:24.696033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c30121-0c26-4aa7-acee-197689ed55c8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:24.704176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c30121-0c26-4aa7-acee-197689ed55c8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:24.705992 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c30121-0c26-4aa7-acee-197689ed55c8 admin admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:24.714584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 9 15:01:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 15:01:24.720265 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c67c12-500b-48db-88a3-44c39e325ea3 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:24.728948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c67c12-500b-48db-88a3-44c39e325ea3 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:24.733299 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72611320-4018-4203-8d0d-f1d65962901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3f51f62b70d42e281dc8a352a6f16d6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:24.734141 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72611320-4018-4203-8d0d-f1d65962901f admin admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:24.734420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72611320-4018-4203-8d0d-f1d65962901f admin admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:24.734694 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72611320-4018-4203-8d0d-f1d65962901f admin admin] PATH_INFO: `/v3/users/c980ae99c7444e0390a8a5607832c346` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:24.736078 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72611320-4018-4203-8d0d-f1d65962901f admin admin] RBAC: Authorizing `identity:get_user(user_id=c980ae99c7444e0390a8a5607832c346)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:24.737923 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72611320-4018-4203-8d0d-f1d65962901f admin admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:24.739463 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon May 9 15:01:24 2022] GET /identity/v3/users/c980ae99c7444e0390a8a5607832c346 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:27.008100 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d9111-fc92-4185-9188-fc557890597d None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.008611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d9111-fc92-4185-9188-fc557890597d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.008723 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d9111-fc92-4185-9188-fc557890597d None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.010117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:27.015872 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c11edf-c7b7-4aa0-83ca-fe43fd662ba3 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.016299 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c11edf-c7b7-4aa0-83ca-fe43fd662ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.016497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c11edf-c7b7-4aa0-83ca-fe43fd662ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.038351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9a29-3777-409a-bbd6-75a41dc69101 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.038636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9a29-3777-409a-bbd6-75a41dc69101 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.038917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea9a29-3777-409a-bbd6-75a41dc69101 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.040299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:27.046417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.046699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.046973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.088506 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG passlib.handlers.bcrypt [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80060) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 15:01:27.088933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG passlib.handlers.bcrypt [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80060) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 15:01:27.136135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:27.137802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:27.140421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-83c11edf-c7b7-4aa0-83ca-fe43fd662ba3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:27.141584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-83c11edf-c7b7-4aa0-83ca-fe43fd662ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:27.219305 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:27.234626 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:27.242521 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c11edf-c7b7-4aa0-83ca-fe43fd662ba3 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:27.256731 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:27.337953 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c11edf-c7b7-4aa0-83ca-fe43fd662ba3 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:27.349962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e12279-17c7-438c-b0ea-4b3d3185c3b3 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:27.370355 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3eacc-6c8a-40bb-b6de-02a351d8aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:27.371149 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3eacc-6c8a-40bb-b6de-02a351d8aa03 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.371437 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3eacc-6c8a-40bb-b6de-02a351d8aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.371717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3eacc-6c8a-40bb-b6de-02a351d8aa03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.381917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174611-2b63-4d9c-940e-348acc6a8649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:27.382807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174611-2b63-4d9c-940e-348acc6a8649 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.382913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3eacc-6c8a-40bb-b6de-02a351d8aa03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:27.383089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174611-2b63-4d9c-940e-348acc6a8649 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.383362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174611-2b63-4d9c-940e-348acc6a8649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-2fc3eacc-6c8a-40bb-b6de-02a351d8aa03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:27.393928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:27.394957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:27.396114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:01:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:27.400862 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3eacc-6c8a-40bb-b6de-02a351d8aa03 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:27.449394 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f174611-2b63-4d9c-940e-348acc6a8649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:27.450920 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f174611-2b63-4d9c-940e-348acc6a8649 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:27.494106 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3eacc-6c8a-40bb-b6de-02a351d8aa03 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:27.526748 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:27.527392 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.527675 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.527950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.528700 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:27.529005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:27.530452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:27.541699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:27.542919 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:01:27.547119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:27.547921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174611-2b63-4d9c-940e-348acc6a8649 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:27.641124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f485c8f-a653-4a4e-933b-7c9f5ecb4d4c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:27.647588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174611-2b63-4d9c-940e-348acc6a8649 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:27.674097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18427b4-a5b8-4558-be72-a9920f3bf6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:27.674955 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18427b4-a5b8-4558-be72-a9920f3bf6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.675242 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18427b4-a5b8-4558-be72-a9920f3bf6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.675518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18427b4-a5b8-4558-be72-a9920f3bf6a1 None admin] PATH_INFO: `/v3/users/c980ae99c7444e0390a8a5607832c346` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.676690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18427b4-a5b8-4558-be72-a9920f3bf6a1 None admin] RBAC: Authorizing `identity:get_user(user_id=c980ae99c7444e0390a8a5607832c346)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:27.678171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18427b4-a5b8-4558-be72-a9920f3bf6a1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:27.678998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:27.679971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:27 2022] GET /identity/v3/users/c980ae99c7444e0390a8a5607832c346 => generated 247 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:27.680074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.680074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.680309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.685067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18427b4-a5b8-4558-be72-a9920f3bf6a1 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:27.785486 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18427b4-a5b8-4558-be72-a9920f3bf6a1 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:27.804987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:27.806478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:27.818893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcbfb4-176d-4025-9d0d-e3c6c993fea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:27.819674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbfb4-176d-4025-9d0d-e3c6c993fea8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.819956 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbfb4-176d-4025-9d0d-e3c6c993fea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.820229 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbfb4-176d-4025-9d0d-e3c6c993fea8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.821430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcbfb4-176d-4025-9d0d-e3c6c993fea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:27.822849 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcbfb4-176d-4025-9d0d-e3c6c993fea8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:27.824432 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:01:27.827826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:27.828764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:27.830056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:01:27.830056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:01:27.830056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:01:27.830056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:01:27.830056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:01:27.830816 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcbfb4-176d-4025-9d0d-e3c6c993fea8 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:27.834587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:27.924127 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcbfb4-176d-4025-9d0d-e3c6c993fea8 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:27.927086 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63933a21-1c6a-43b6-931c-8aaed884e2ca None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:27.956048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2c603-2455-4559-b17c-a310778ebc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:27.956690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2c603-2455-4559-b17c-a310778ebc8b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.956976 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2c603-2455-4559-b17c-a310778ebc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.957282 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2c603-2455-4559-b17c-a310778ebc8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.958124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2c603-2455-4559-b17c-a310778ebc8b None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.domain.id=default, user.id=c980ae99c7444e0390a8a5607832c346) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:27.958282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b78cd1-c1fa-44fc-a88f-9b8dd66f6b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:27.958442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2c603-2455-4559-b17c-a310778ebc8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:27.958923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78cd1-c1fa-44fc-a88f-9b8dd66f6b4c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:27.959203 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78cd1-c1fa-44fc-a88f-9b8dd66f6b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:27.959477 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78cd1-c1fa-44fc-a88f-9b8dd66f6b4c None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:27.959904 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2c603-2455-4559-b17c-a310778ebc8b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:27.960194 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b78cd1-c1fa-44fc-a88f-9b8dd66f6b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:27.960499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b78cd1-c1fa-44fc-a88f-9b8dd66f6b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:27.961882 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b78cd1-c1fa-44fc-a88f-9b8dd66f6b4c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:27.966439 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 9 15:01:27 2022] GET /identity/v3/role_assignments?user.id=c980ae99c7444e0390a8a5607832c346&scope.domain.id=default&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 233 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:27.980587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:30.789079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e52b0-de6f-4cef-8708-38799259a12f None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:30.789079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e52b0-de6f-4cef-8708-38799259a12f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:30.789079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e52b0-de6f-4cef-8708-38799259a12f None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:30.790445 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:30.796784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d22d7-d4b3-477f-b55f-336a2f995bc6 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:30.797691 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d22d7-d4b3-477f-b55f-336a2f995bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:30.798191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d22d7-d4b3-477f-b55f-336a2f995bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:30.821807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-548d22d7-d4b3-477f-b55f-336a2f995bc6 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:30.823435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-548d22d7-d4b3-477f-b55f-336a2f995bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:30.913748 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:30.948535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e52b0-de6f-4cef-8708-38799259a12f None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:30.989757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e82c87-cd82-496c-9e2f-172e5f8b8700 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:30.990388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e82c87-cd82-496c-9e2f-172e5f8b8700 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:30.990989 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e82c87-cd82-496c-9e2f-172e5f8b8700 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:30.992545 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:30.997524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f36aa-f4a8-4e82-ba95-900a96a7a002 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:30.998041 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f36aa-f4a8-4e82-ba95-900a96a7a002 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:30.998561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f36aa-f4a8-4e82-ba95-900a96a7a002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.026405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-a70f36aa-f4a8-4e82-ba95-900a96a7a002 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:31.027934 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-a70f36aa-f4a8-4e82-ba95-900a96a7a002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:31.137327 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e52b0-de6f-4cef-8708-38799259a12f None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:31.149078 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:31.150412 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade201-e73c-4b3a-baa7-f961639634c5 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.150703 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade201-e73c-4b3a-baa7-f961639634c5 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.150983 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aade201-e73c-4b3a-baa7-f961639634c5 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.152499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 14/26] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:31.153713 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699dc58-637e-4781-880d-2fc6677e79c1 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.153901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699dc58-637e-4781-880d-2fc6677e79c1 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.154137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699dc58-637e-4781-880d-2fc6677e79c1 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.159600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 15/27] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:31.161090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b1f74-40ae-4d40-bb1f-08f75562061d None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.161198 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b1f74-40ae-4d40-bb1f-08f75562061d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.161415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b1f74-40ae-4d40-bb1f-08f75562061d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.179349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cfd0f-5ff6-49a1-9259-d54412eeb4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:31.180101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cfd0f-5ff6-49a1-9259-d54412eeb4f1 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.180383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cfd0f-5ff6-49a1-9259-d54412eeb4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.180795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cfd0f-5ff6-49a1-9259-d54412eeb4f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.181702 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cfd0f-5ff6-49a1-9259-d54412eeb4f1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:31.183652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cfd0f-5ff6-49a1-9259-d54412eeb4f1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:31.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-f63b1f74-40ae-4d40-bb1f-08f75562061d None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:31.190699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-f63b1f74-40ae-4d40-bb1f-08f75562061d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:31.264211 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 16/28] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:31.265734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee5295-c2f4-4c30-b417-e590e56f8c6b None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.266093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee5295-c2f4-4c30-b417-e590e56f8c6b None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.266462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee5295-c2f4-4c30-b417-e590e56f8c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.282919 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-21ee5295-c2f4-4c30-b417-e590e56f8c6b None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:31.284171 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-21ee5295-c2f4-4c30-b417-e590e56f8c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:31.321690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 13/29] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:01:30 2022] POST /identity/v3/roles => generated 219 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:31.323072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cfd0f-5ff6-49a1-9259-d54412eeb4f1 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:31.407025 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 17/30] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:31.408332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee5295-c2f4-4c30-b417-e590e56f8c6b None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:31.434457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cfd0f-5ff6-49a1-9259-d54412eeb4f1 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:31.471472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017b1ad-5436-486e-a018-c791ec1e9cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:31.472730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017b1ad-5436-486e-a018-c791ec1e9cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.473275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017b1ad-5436-486e-a018-c791ec1e9cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.473798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017b1ad-5436-486e-a018-c791ec1e9cd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.485590 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017b1ad-5436-486e-a018-c791ec1e9cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-f017b1ad-5436-486e-a018-c791ec1e9cd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:31.494090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:31.495012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:31.495777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:31.498455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 14/31] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:01:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:31.499572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017b1ad-5436-486e-a018-c791ec1e9cd1 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:31.542344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee5295-c2f4-4c30-b417-e590e56f8c6b None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:31.586728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee66da2-1914-4e1a-b8c0-410bf52f0f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:31.587428 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee66da2-1914-4e1a-b8c0-410bf52f0f0c None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.587708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee66da2-1914-4e1a-b8c0-410bf52f0f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.587978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee66da2-1914-4e1a-b8c0-410bf52f0f0c None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.588742 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee66da2-1914-4e1a-b8c0-410bf52f0f0c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:31.590159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee66da2-1914-4e1a-b8c0-410bf52f0f0c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:31.610450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017b1ad-5436-486e-a018-c791ec1e9cd1 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:31.650077 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:31.651286 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.651771 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.652267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.665444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:31.667488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:01:31.688857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:31.689810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:31.690632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:01:31.690632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:01:31.690632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:01:31.690632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:31.693235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 15/32] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:31.694324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:31.781547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 18/33] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:01:31 2022] POST /identity/v3/roles => generated 225 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:31.783165 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee66da2-1914-4e1a-b8c0-410bf52f0f0c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:31.815846 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0162d2f-df50-46d6-b15a-62650dd3e7c0 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:31.853423 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:31.854517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.855004 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.855523 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.856632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:31.857166 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:31.858882 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:31.868399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 16/34] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:31.876788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:31.902431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee66da2-1914-4e1a-b8c0-410bf52f0f0c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:31.942012 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5f644-a5fb-44c0-8cb7-38dc36c057dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:31.942556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5f644-a5fb-44c0-8cb7-38dc36c057dd None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.942845 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5f644-a5fb-44c0-8cb7-38dc36c057dd None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.943127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5f644-a5fb-44c0-8cb7-38dc36c057dd None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.943865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5f644-a5fb-44c0-8cb7-38dc36c057dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:31.944174 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5f644-a5fb-44c0-8cb7-38dc36c057dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:31.945714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5f644-a5fb-44c0-8cb7-38dc36c057dd None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:31.958386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 19/35] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:31.982272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264c64-298f-4945-8a9e-4a4f373ffe6b None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:31.982958 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264c64-298f-4945-8a9e-4a4f373ffe6b None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:31.983429 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee264c64-298f-4945-8a9e-4a4f373ffe6b None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:31.985023 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 20/36] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:31.986042 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee264c64-298f-4945-8a9e-4a4f373ffe6b None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:31.995968 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083fd7-9f16-4fb5-bdb9-7337bcff164b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:32.034299 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e5342-2451-460f-9f0f-025728b43d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:32.035422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e5342-2451-460f-9f0f-025728b43d8d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.035540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e5342-2451-460f-9f0f-025728b43d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.035790 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e5342-2451-460f-9f0f-025728b43d8d None admin] PATH_INFO: `/v3/users/c980ae99c7444e0390a8a5607832c346` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.037465 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e5342-2451-460f-9f0f-025728b43d8d None admin] RBAC: Authorizing `identity:get_user(user_id=c980ae99c7444e0390a8a5607832c346)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:32.039021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e5342-2451-460f-9f0f-025728b43d8d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:32.041063 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 17/37] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:31 2022] GET /identity/v3/users/c980ae99c7444e0390a8a5607832c346 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:32.042562 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150950d-5f57-4ebe-b8f3-a7fc85921d57 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.042758 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150950d-5f57-4ebe-b8f3-a7fc85921d57 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.042979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150950d-5f57-4ebe-b8f3-a7fc85921d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.062442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-1150950d-5f57-4ebe-b8f3-a7fc85921d57 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:32.064133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-1150950d-5f57-4ebe-b8f3-a7fc85921d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:32.125188 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee264c64-298f-4945-8a9e-4a4f373ffe6b None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:32.144117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 18/38] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:32.144979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150950d-5f57-4ebe-b8f3-a7fc85921d57 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:32.177635 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecea44-44ae-4b2c-8a57-b43733f6ac2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:32.178425 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecea44-44ae-4b2c-8a57-b43733f6ac2e None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.178652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecea44-44ae-4b2c-8a57-b43733f6ac2e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.178925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecea44-44ae-4b2c-8a57-b43733f6ac2e None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.180465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecea44-44ae-4b2c-8a57-b43733f6ac2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:32.181976 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecea44-44ae-4b2c-8a57-b43733f6ac2e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:32.217643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 21/39] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:01:31 2022] POST /identity/v3/projects => generated 301 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:32.217643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecea44-44ae-4b2c-8a57-b43733f6ac2e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:32.269422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150950d-5f57-4ebe-b8f3-a7fc85921d57 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:32.312346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402ca96-7442-4b56-8b1c-cac54a178049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:32.313858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402ca96-7442-4b56-8b1c-cac54a178049 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.314296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402ca96-7442-4b56-8b1c-cac54a178049 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.314772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402ca96-7442-4b56-8b1c-cac54a178049 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.317392 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402ca96-7442-4b56-8b1c-cac54a178049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:32.319201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402ca96-7442-4b56-8b1c-cac54a178049 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:32.321322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 19/40] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:32.329161 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402ca96-7442-4b56-8b1c-cac54a178049 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:32.358217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecea44-44ae-4b2c-8a57-b43733f6ac2e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:32.405278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a31ca4b-d02f-44dc-b488-e16d15dbefed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:32.406306 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31ca4b-d02f-44dc-b488-e16d15dbefed None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.406771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31ca4b-d02f-44dc-b488-e16d15dbefed None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.407224 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31ca4b-d02f-44dc-b488-e16d15dbefed None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.408166 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31ca4b-d02f-44dc-b488-e16d15dbefed None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:32.409810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a31ca4b-d02f-44dc-b488-e16d15dbefed None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:32.427914 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 22/41] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:01:32 2022] POST /identity/v3/roles => generated 223 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:32.436314 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cebcb-2349-4d99-aae3-b992a44d2a2c None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.436618 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cebcb-2349-4d99-aae3-b992a44d2a2c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.436882 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cebcb-2349-4d99-aae3-b992a44d2a2c None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.438530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 23/42] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:32.444786 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc6c8-3108-495a-b1c4-080d92987711 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.444786 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc6c8-3108-495a-b1c4-080d92987711 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.445167 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bc6c8-3108-495a-b1c4-080d92987711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.446035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402ca96-7442-4b56-8b1c-cac54a178049 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:32.465233 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-bd3bc6c8-3108-495a-b1c4-080d92987711 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:32.466771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-bd3bc6c8-3108-495a-b1c4-080d92987711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:32.499528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723eaba-26d9-423f-851f-7f45e6973a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:32.500550 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723eaba-26d9-423f-851f-7f45e6973a7a None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.500963 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723eaba-26d9-423f-851f-7f45e6973a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.501427 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723eaba-26d9-423f-851f-7f45e6973a7a None admin] PATH_INFO: `/v3/domains/default/users/c980ae99c7444e0390a8a5607832c346/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.503777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723eaba-26d9-423f-851f-7f45e6973a7a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c980ae99c7444e0390a8a5607832c346, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:32.505747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723eaba-26d9-423f-851f-7f45e6973a7a None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:32.527518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 20/43] 10.222.0.13 () {58 vars in 1361 bytes} [Mon May 9 15:01:32 2022] PUT /identity/v3/domains/default/users/c980ae99c7444e0390a8a5607832c346/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:01:32.559465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 24/44] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:32.587245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723eaba-26d9-423f-851f-7f45e6973a7a None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:32.698217 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723eaba-26d9-423f-851f-7f45e6973a7a None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:32.733212 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdf8f9-f929-4d51-8cf3-7a8f6ab2089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:32.734014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdf8f9-f929-4d51-8cf3-7a8f6ab2089d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.734299 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdf8f9-f929-4d51-8cf3-7a8f6ab2089d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.734574 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdf8f9-f929-4d51-8cf3-7a8f6ab2089d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.735744 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdf8f9-f929-4d51-8cf3-7a8f6ab2089d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:32.737221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdf8f9-f929-4d51-8cf3-7a8f6ab2089d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:32.738841 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 21/45] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:32.744362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bc6c8-3108-495a-b1c4-080d92987711 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:32.839071 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bc6c8-3108-495a-b1c4-080d92987711 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:32.870353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8f717-e574-4b13-b0e7-e563cb82570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:32.871016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f717-e574-4b13-b0e7-e563cb82570e None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:32.871299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f717-e574-4b13-b0e7-e563cb82570e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:32.871577 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f717-e574-4b13-b0e7-e563cb82570e None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:32.872285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8f717-e574-4b13-b0e7-e563cb82570e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:32.874269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8f717-e574-4b13-b0e7-e563cb82570e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:32.898537 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 25/46] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:01:32 2022] POST /identity/v3/projects => generated 312 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:34.751155 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2656f-6644-4c2d-a70e-b95ce7d6f9d6 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:34.751667 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2656f-6644-4c2d-a70e-b95ce7d6f9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:34.751772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2656f-6644-4c2d-a70e-b95ce7d6f9d6 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:34.753233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 22/47] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:34.758491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ac21-b3b6-4c0f-88b5-0086956a68b3 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:34.758877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ac21-b3b6-4c0f-88b5-0086956a68b3 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:34.759170 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ac21-b3b6-4c0f-88b5-0086956a68b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:34.775497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-00e5ac21-b3b6-4c0f-88b5-0086956a68b3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:34.776660 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-00e5ac21-b3b6-4c0f-88b5-0086956a68b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:34.841127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 26/48] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:34.865515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2656f-6644-4c2d-a70e-b95ce7d6f9d6 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:34.963858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2656f-6644-4c2d-a70e-b95ce7d6f9d6 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:34.998215 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22877a2-76ed-4edb-9c23-6ac5d607ad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:34.999203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22877a2-76ed-4edb-9c23-6ac5d607ad26 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:34.999492 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22877a2-76ed-4edb-9c23-6ac5d607ad26 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:34.999773 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22877a2-76ed-4edb-9c23-6ac5d607ad26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.009882 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22877a2-76ed-4edb-9c23-6ac5d607ad26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-b22877a2-76ed-4edb-9c23-6ac5d607ad26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:35.017516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:35.018579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:35.019511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 23/49] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:35.026217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ac21-b3b6-4c0f-88b5-0086956a68b3 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:35.120331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ac21-b3b6-4c0f-88b5-0086956a68b3 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:35.137421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66206242-ae63-468c-b846-9b6725116c1c None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:35.137713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66206242-ae63-468c-b846-9b6725116c1c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:35.138006 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66206242-ae63-468c-b846-9b6725116c1c None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.139886 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 24/50] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:35.144827 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb8a94-8f17-48a3-807e-82e9879f4bae None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:35.145149 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb8a94-8f17-48a3-807e-82e9879f4bae None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:35.145437 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb8a94-8f17-48a3-807e-82e9879f4bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.153596 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:35.154260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:35.155112 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:35.155391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.156153 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:35.156467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:35.157892 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:35.162432 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-7beb8a94-8f17-48a3-807e-82e9879f4bae None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:35.163626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-7beb8a94-8f17-48a3-807e-82e9879f4bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:35.168950 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 27/51] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:35.174264 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:35.230339 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 25/52] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:35.253140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb8a94-8f17-48a3-807e-82e9879f4bae None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:35.270640 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050396c3-b716-481c-9816-ab0d63843f35 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:35.301870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a159-365b-49e0-98f5-1d9cdf301069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:35.302730 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a159-365b-49e0-98f5-1d9cdf301069 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:35.303013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a159-365b-49e0-98f5-1d9cdf301069 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:35.303288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59a159-365b-49e0-98f5-1d9cdf301069 None admin] PATH_INFO: `/v3/users/c980ae99c7444e0390a8a5607832c346` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.304702 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59a159-365b-49e0-98f5-1d9cdf301069 None admin] RBAC: Authorizing `identity:get_user(user_id=c980ae99c7444e0390a8a5607832c346)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:35.306169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59a159-365b-49e0-98f5-1d9cdf301069 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:35.307785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 28/53] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:35 2022] GET /identity/v3/users/c980ae99c7444e0390a8a5607832c346 => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:35.313024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a159-365b-49e0-98f5-1d9cdf301069 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:35.346875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb8a94-8f17-48a3-807e-82e9879f4bae None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:35.378488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c78145-58f6-4041-aa09-a458bfe49e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:35.379263 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78145-58f6-4041-aa09-a458bfe49e03 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:35.379546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78145-58f6-4041-aa09-a458bfe49e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:35.379824 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78145-58f6-4041-aa09-a458bfe49e03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.381288 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c78145-58f6-4041-aa09-a458bfe49e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:35.382759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c78145-58f6-4041-aa09-a458bfe49e03 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:35.384361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 26/54] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:35.390171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c78145-58f6-4041-aa09-a458bfe49e03 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:35.405602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59a159-365b-49e0-98f5-1d9cdf301069 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:35.436120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b9325-52b2-4016-a756-1f6793ec38e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:35.436892 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9325-52b2-4016-a756-1f6793ec38e2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:35.437209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9325-52b2-4016-a756-1f6793ec38e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:35.437487 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9325-52b2-4016-a756-1f6793ec38e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.438645 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b9325-52b2-4016-a756-1f6793ec38e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:35.440067 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b9325-52b2-4016-a756-1f6793ec38e2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:35.441701 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 29/55] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:35.447584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b9325-52b2-4016-a756-1f6793ec38e2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:35.483788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c78145-58f6-4041-aa09-a458bfe49e03 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:35.516094 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3eba1d-c27f-47cc-b34f-3f7e0fa8133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:35.516737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eba1d-c27f-47cc-b34f-3f7e0fa8133f None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:35.517020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eba1d-c27f-47cc-b34f-3f7e0fa8133f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:35.517324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3eba1d-c27f-47cc-b34f-3f7e0fa8133f None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.518022 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3eba1d-c27f-47cc-b34f-3f7e0fa8133f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:35.519464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3eba1d-c27f-47cc-b34f-3f7e0fa8133f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:35.541693 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b9325-52b2-4016-a756-1f6793ec38e2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:35.545472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 27/56] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 9 15:01:35 2022] POST /identity/v3/projects => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:01:35.575318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20483c06-311e-437d-adde-cd28172f3f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:35.575995 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20483c06-311e-437d-adde-cd28172f3f8e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:35.576899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20483c06-311e-437d-adde-cd28172f3f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:35.577215 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20483c06-311e-437d-adde-cd28172f3f8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:35.578075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20483c06-311e-437d-adde-cd28172f3f8e None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.domain.id=default, user.id=c980ae99c7444e0390a8a5607832c346) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:35.578391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20483c06-311e-437d-adde-cd28172f3f8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:35.579875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20483c06-311e-437d-adde-cd28172f3f8e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:35.589953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 30/57] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 9 15:01:35 2022] GET /identity/v3/role_assignments?user.id=c980ae99c7444e0390a8a5607832c346&scope.domain.id=default&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:37.820401 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef45ce-1085-470e-a1c7-8277c3d40376 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:37.820917 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef45ce-1085-470e-a1c7-8277c3d40376 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:37.820999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef45ce-1085-470e-a1c7-8277c3d40376 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:37.822365 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 28/58] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:37.827543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4957-9b3c-487b-ae82-54101de90597 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:37.827914 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4957-9b3c-487b-ae82-54101de90597 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:37.828112 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4957-9b3c-487b-ae82-54101de90597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:37.844782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-937d4957-9b3c-487b-ae82-54101de90597 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:37.845998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-937d4957-9b3c-487b-ae82-54101de90597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:37.908465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 31/59] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:37.932701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef45ce-1085-470e-a1c7-8277c3d40376 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:38.029002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef45ce-1085-470e-a1c7-8277c3d40376 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:38.059893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59828fd1-ddef-4910-95d7-1d6859bb65c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:38.061220 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59828fd1-ddef-4910-95d7-1d6859bb65c8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:38.061505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59828fd1-ddef-4910-95d7-1d6859bb65c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:38.061780 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59828fd1-ddef-4910-95d7-1d6859bb65c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:38.062963 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59828fd1-ddef-4910-95d7-1d6859bb65c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:38.064407 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59828fd1-ddef-4910-95d7-1d6859bb65c8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:38.066021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 29/60] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:38.071357 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d4957-9b3c-487b-ae82-54101de90597 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:38.165952 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d4957-9b3c-487b-ae82-54101de90597 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:38.199488 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc288b8-5475-4dfa-824f-8f6c56483a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:38.200153 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc288b8-5475-4dfa-824f-8f6c56483a36 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:38.200434 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc288b8-5475-4dfa-824f-8f6c56483a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:38.200711 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc288b8-5475-4dfa-824f-8f6c56483a36 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:38.201469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc288b8-5475-4dfa-824f-8f6c56483a36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:38.202916 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc288b8-5475-4dfa-824f-8f6c56483a36 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:38.225832 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 32/61] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:01:38 2022] POST /identity/v3/users => generated 275 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:41.379580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb86008-4780-4dc0-a3cd-158410ecb802 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:41.380651 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb86008-4780-4dc0-a3cd-158410ecb802 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:41.381113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb86008-4780-4dc0-a3cd-158410ecb802 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:41.382617 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 30/62] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:41.387870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393b06d-4846-41dd-87f9-56740240f04b None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:41.389145 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393b06d-4846-41dd-87f9-56740240f04b None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:41.389560 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393b06d-4846-41dd-87f9-56740240f04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:41.410134 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-c393b06d-4846-41dd-87f9-56740240f04b None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:41.411518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-c393b06d-4846-41dd-87f9-56740240f04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:41.498558 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 33/63] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:41.530666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb86008-4780-4dc0-a3cd-158410ecb802 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:41.658669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb86008-4780-4dc0-a3cd-158410ecb802 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:41.696614 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e4d83-892a-4c22-ac12-a0fe1d45dd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:41.697442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e4d83-892a-4c22-ac12-a0fe1d45dd63 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:41.697734 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e4d83-892a-4c22-ac12-a0fe1d45dd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:41.698016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e4d83-892a-4c22-ac12-a0fe1d45dd63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:41.718059 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e4d83-892a-4c22-ac12-a0fe1d45dd63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-f09e4d83-892a-4c22-ac12-a0fe1d45dd63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:41.725880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:41.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:41.727782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 31/64] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:41.733673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393b06d-4846-41dd-87f9-56740240f04b None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:41.739865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781af212-6f11-4583-8aff-685eefde6edc None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:41.739865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781af212-6f11-4583-8aff-685eefde6edc None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:41.739865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781af212-6f11-4583-8aff-685eefde6edc None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:41.739865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 32/65] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:41.751152 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637c9e-6e94-4677-a9bd-f941f558c66f None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:41.751667 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637c9e-6e94-4677-a9bd-f941f558c66f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:41.752085 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637c9e-6e94-4677-a9bd-f941f558c66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:41.784535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-9f637c9e-6e94-4677-a9bd-f941f558c66f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:41.785780 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-9f637c9e-6e94-4677-a9bd-f941f558c66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:41.889585 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 33/66] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:41.913122 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f637c9e-6e94-4677-a9bd-f941f558c66f None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:41.931975 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393b06d-4846-41dd-87f9-56740240f04b None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:41.991847 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af92b7d-bd81-4529-a429-35ef3eb86d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:41.992521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af92b7d-bd81-4529-a429-35ef3eb86d9b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:41.992804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af92b7d-bd81-4529-a429-35ef3eb86d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:41.993110 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af92b7d-bd81-4529-a429-35ef3eb86d9b None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:41.993872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af92b7d-bd81-4529-a429-35ef3eb86d9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:41.994187 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af92b7d-bd81-4529-a429-35ef3eb86d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:41.995577 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af92b7d-bd81-4529-a429-35ef3eb86d9b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:42.004878 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 34/67] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:42.006497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37986-951c-494a-8390-8febbf8b3381 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.006790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37986-951c-494a-8390-8febbf8b3381 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.007070 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a37986-951c-494a-8390-8febbf8b3381 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.008433 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 35/68] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:42.013371 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a37986-951c-494a-8390-8febbf8b3381 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.018627 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f637c9e-6e94-4677-a9bd-f941f558c66f None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.059810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d351a1-a6ac-401d-bf69-fddd6e6dcc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.061290 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d351a1-a6ac-401d-bf69-fddd6e6dcc2d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.061579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d351a1-a6ac-401d-bf69-fddd6e6dcc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.061857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d351a1-a6ac-401d-bf69-fddd6e6dcc2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.073849 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d351a1-a6ac-401d-bf69-fddd6e6dcc2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-40d351a1-a6ac-401d-bf69-fddd6e6dcc2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:42.081698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:42.082638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:42.083413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:42.087870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 34/69] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:01:42.089477 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced464-1d06-47bc-bc34-8afea24fbf10 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.089771 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced464-1d06-47bc-bc34-8afea24fbf10 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.090054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced464-1d06-47bc-bc34-8afea24fbf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.107900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-14ced464-1d06-47bc-bc34-8afea24fbf10 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:42.109130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-14ced464-1d06-47bc-bc34-8afea24fbf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:42.139590 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a37986-951c-494a-8390-8febbf8b3381 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.176031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d3634-2971-4f34-9e54-48b03c5fb789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.177150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d3634-2971-4f34-9e54-48b03c5fb789 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.177596 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d3634-2971-4f34-9e54-48b03c5fb789 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.178073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d3634-2971-4f34-9e54-48b03c5fb789 None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.193874 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d3634-2971-4f34-9e54-48b03c5fb789 None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:42.195567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d3634-2971-4f34-9e54-48b03c5fb789 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:42.197514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 36/70] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:42.198471 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d3634-2971-4f34-9e54-48b03c5fb789 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.205946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 35/71] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:42.213180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ced464-1d06-47bc-bc34-8afea24fbf10 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.308831 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ced464-1d06-47bc-bc34-8afea24fbf10 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.311083 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d3634-2971-4f34-9e54-48b03c5fb789 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.341941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d004b7-0608-4f27-ad09-f05d61345551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.342596 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d004b7-0608-4f27-ad09-f05d61345551 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.342880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d004b7-0608-4f27-ad09-f05d61345551 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.343157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d004b7-0608-4f27-ad09-f05d61345551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.343244 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6313c4f-2d58-4e19-a9c3-b0315137789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.343830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313c4f-2d58-4e19-a9c3-b0315137789a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.344125 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313c4f-2d58-4e19-a9c3-b0315137789a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.344398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313c4f-2d58-4e19-a9c3-b0315137789a None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.345188 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6313c4f-2d58-4e19-a9c3-b0315137789a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:42.345501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6313c4f-2d58-4e19-a9c3-b0315137789a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:42.346885 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6313c4f-2d58-4e19-a9c3-b0315137789a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:42.354582 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 37/72] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:42.359607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055ffb-40e7-4218-9a2e-80b94f53db70 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.359607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055ffb-40e7-4218-9a2e-80b94f53db70 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.359607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055ffb-40e7-4218-9a2e-80b94f53db70 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.359607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 38/73] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:42.359607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3055ffb-40e7-4218-9a2e-80b94f53db70 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.411468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d004b7-0608-4f27-ad09-f05d61345551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:42.412964 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d004b7-0608-4f27-ad09-f05d61345551 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:42.457197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3055ffb-40e7-4218-9a2e-80b94f53db70 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.487774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffca9b-9df6-46f6-a57a-0e2887a65a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.488565 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffca9b-9df6-46f6-a57a-0e2887a65a72 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.488847 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffca9b-9df6-46f6-a57a-0e2887a65a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.489146 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffca9b-9df6-46f6-a57a-0e2887a65a72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.500631 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffca9b-9df6-46f6-a57a-0e2887a65a72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:42.504037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 36/74] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:42.505608 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660fe4b-5d54-4145-9dce-ad095bdd9dff None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.505898 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660fe4b-5d54-4145-9dce-ad095bdd9dff None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.506174 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660fe4b-5d54-4145-9dce-ad095bdd9dff None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.507614 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 37/75] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:42.508113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660fe4b-5d54-4145-9dce-ad095bdd9dff None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-37ffca9b-9df6-46f6-a57a-0e2887a65a72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:42.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:42.515736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:01:42.516533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 39/76] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:42.517663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c28c-a898-4841-a85c-82a89dfca26c None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.517663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c28c-a898-4841-a85c-82a89dfca26c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.517884 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c28c-a898-4841-a85c-82a89dfca26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.534168 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-b155c28c-a898-4841-a85c-82a89dfca26c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:42.535394 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-b155c28c-a898-4841-a85c-82a89dfca26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:42.597264 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 40/77] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:42.598044 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155c28c-a898-4841-a85c-82a89dfca26c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.603513 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660fe4b-5d54-4145-9dce-ad095bdd9dff None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.637353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7e188-1266-4d2c-9b5b-7873f8639ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.638232 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e188-1266-4d2c-9b5b-7873f8639ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.639105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e188-1266-4d2c-9b5b-7873f8639ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.639393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e188-1266-4d2c-9b5b-7873f8639ec0 None admin] PATH_INFO: `/v3/users/c980ae99c7444e0390a8a5607832c346` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.640606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7e188-1266-4d2c-9b5b-7873f8639ec0 None admin] RBAC: Authorizing `identity:get_user(user_id=c980ae99c7444e0390a8a5607832c346)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:42.642190 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7e188-1266-4d2c-9b5b-7873f8639ec0 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:42.643802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 38/78] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/users/c980ae99c7444e0390a8a5607832c346 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:42.646237 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005122b3-cc3b-4151-b5ec-3654dc4e62e7 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.646522 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005122b3-cc3b-4151-b5ec-3654dc4e62e7 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.646797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005122b3-cc3b-4151-b5ec-3654dc4e62e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.662198 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-005122b3-cc3b-4151-b5ec-3654dc4e62e7 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:42.663362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-005122b3-cc3b-4151-b5ec-3654dc4e62e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:42.692326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155c28c-a898-4841-a85c-82a89dfca26c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.724504 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1569302-acb0-4ca3-8d36-2f0c255d9df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.725323 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1569302-acb0-4ca3-8d36-2f0c255d9df1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.725742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 39/79] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:42.725846 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1569302-acb0-4ca3-8d36-2f0c255d9df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.725937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1569302-acb0-4ca3-8d36-2f0c255d9df1 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.726378 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005122b3-cc3b-4151-b5ec-3654dc4e62e7 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.727495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1569302-acb0-4ca3-8d36-2f0c255d9df1 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:42.728953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1569302-acb0-4ca3-8d36-2f0c255d9df1 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:42.730838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 41/80] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:42.734259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1569302-acb0-4ca3-8d36-2f0c255d9df1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.818114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005122b3-cc3b-4151-b5ec-3654dc4e62e7 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.829733 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1569302-acb0-4ca3-8d36-2f0c255d9df1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.850159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.850816 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.851102 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.851378 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.852125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:42.852435 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:42.853849 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:42.861420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b4a50-2979-4108-b2f3-b3adb2f42217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.861559 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 40/81] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:42.862207 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.863136 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b4a50-2979-4108-b2f3-b3adb2f42217 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.863424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b4a50-2979-4108-b2f3-b3adb2f42217 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.863724 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b4a50-2979-4108-b2f3-b3adb2f42217 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:42.874229 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b4a50-2979-4108-b2f3-b3adb2f42217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f57b4a50-2979-4108-b2f3-b3adb2f42217 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:42.882096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:42.883045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:01:42.883828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 42/82] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:42.884463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b4a50-2979-4108-b2f3-b3adb2f42217 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:42.957923 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fafd8-3916-4285-94b0-8e957b89fa7d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.976998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b4a50-2979-4108-b2f3-b3adb2f42217 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:42.988932 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b376f-ebbb-428b-ba1e-c4e6a23644cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:42.989591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b376f-ebbb-428b-ba1e-c4e6a23644cf None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:42.989874 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b376f-ebbb-428b-ba1e-c4e6a23644cf None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:42.990146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b376f-ebbb-428b-ba1e-c4e6a23644cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.009192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.009837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.010123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.010406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.011236 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] RBAC: Adding query filter params (role.id=67832478d7b8428d996975409814f845, scope.project.id=8f180df64cf54d3996588ba6518e4e4c, user.id=6a6136a2abe6402cab688d9bb754142c) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:43.011553 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.013004 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.018455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 43/83] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/role_assignments?user.id=6a6136a2abe6402cab688d9bb754142c&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=67832478d7b8428d996975409814f845 => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:43.019003 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.073094 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b376f-ebbb-428b-ba1e-c4e6a23644cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.074647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b376f-ebbb-428b-ba1e-c4e6a23644cf None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.131968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9361-243e-4586-8814-faac4502c68a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.172117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4e6ed-c702-4a63-9b45-7ff1670e1ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.173014 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4e6ed-c702-4a63-9b45-7ff1670e1ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.173398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4e6ed-c702-4a63-9b45-7ff1670e1ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.173686 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4e6ed-c702-4a63-9b45-7ff1670e1ee7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.174912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4e6ed-c702-4a63-9b45-7ff1670e1ee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.176407 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4e6ed-c702-4a63-9b45-7ff1670e1ee7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.178315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 44/84] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:43.178946 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4e6ed-c702-4a63-9b45-7ff1670e1ee7 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.183749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 41/85] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:43.184838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b376f-ebbb-428b-ba1e-c4e6a23644cf None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.283856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b376f-ebbb-428b-ba1e-c4e6a23644cf None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.290520 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4e6ed-c702-4a63-9b45-7ff1670e1ee7 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.318058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.318718 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.319003 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.319277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.320036 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:43.320325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.321759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.322339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdb3a6-14ac-476b-8cb8-ba02ed5a5389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.323201 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdb3a6-14ac-476b-8cb8-ba02ed5a5389 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.323477 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdb3a6-14ac-476b-8cb8-ba02ed5a5389 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.323749 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdb3a6-14ac-476b-8cb8-ba02ed5a5389 None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.324926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdb3a6-14ac-476b-8cb8-ba02ed5a5389 None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.327283 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdb3a6-14ac-476b-8cb8-ba02ed5a5389 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.329562 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 45/86] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:43.329562 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdb3a6-14ac-476b-8cb8-ba02ed5a5389 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.334275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 42/87] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:43.334812 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.428133 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdb3a6-14ac-476b-8cb8-ba02ed5a5389 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.432484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916e099-38cd-4f5b-8817-8d2bca9174d4 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.460083 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672de83-d8d7-4f12-b880-f5c49e389132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.460839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672de83-d8d7-4f12-b880-f5c49e389132 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.461223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672de83-d8d7-4f12-b880-f5c49e389132 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.461609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672de83-d8d7-4f12-b880-f5c49e389132 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.462257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672de83-d8d7-4f12-b880-f5c49e389132 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.463620 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672de83-d8d7-4f12-b880-f5c49e389132 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.465281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad9caa-0b41-40ee-aafe-47d8a4563b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.466137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad9caa-0b41-40ee-aafe-47d8a4563b92 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.466419 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad9caa-0b41-40ee-aafe-47d8a4563b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.466725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad9caa-0b41-40ee-aafe-47d8a4563b92 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.467916 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad9caa-0b41-40ee-aafe-47d8a4563b92 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.469397 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad9caa-0b41-40ee-aafe-47d8a4563b92 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.475981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 43/88] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:43.475981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad9caa-0b41-40ee-aafe-47d8a4563b92 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.489997 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 46/89] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:01:43 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:43.491135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672de83-d8d7-4f12-b880-f5c49e389132 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.589244 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad9caa-0b41-40ee-aafe-47d8a4563b92 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.606137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672de83-d8d7-4f12-b880-f5c49e389132 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.624852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b0dff-b36e-4d0d-98e1-d49b33fc741d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.625829 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b0dff-b36e-4d0d-98e1-d49b33fc741d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.626116 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b0dff-b36e-4d0d-98e1-d49b33fc741d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.626395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b0dff-b36e-4d0d-98e1-d49b33fc741d None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.627692 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b0dff-b36e-4d0d-98e1-d49b33fc741d None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.629544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b0dff-b36e-4d0d-98e1-d49b33fc741d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.631319 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 44/90] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:43.632187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b0dff-b36e-4d0d-98e1-d49b33fc741d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.645354 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efde9d-487e-4d15-8845-01bcd602ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.646502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efde9d-487e-4d15-8845-01bcd602ea4d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.646988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efde9d-487e-4d15-8845-01bcd602ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.647463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efde9d-487e-4d15-8845-01bcd602ea4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.721516 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efde9d-487e-4d15-8845-01bcd602ea4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.722790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efde9d-487e-4d15-8845-01bcd602ea4d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.740188 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b0dff-b36e-4d0d-98e1-d49b33fc741d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.776541 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.777244 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.777533 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.777808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.778656 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=8f180df64cf54d3996588ba6518e4e4c, user.id=c980ae99c7444e0390a8a5607832c346) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:43.778970 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.781109 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:43.786937 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 45/91] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/role_assignments?user.id=c980ae99c7444e0390a8a5607832c346&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:43.787661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.838671 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 47/92] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:43.846766 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efde9d-487e-4d15-8845-01bcd602ea4d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:43.908780 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb35b1-796c-41ab-b264-a4af8083d992 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.946159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e42fb-07e9-4152-9fc2-01c29b71592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.946811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e42fb-07e9-4152-9fc2-01c29b71592b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.947094 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e42fb-07e9-4152-9fc2-01c29b71592b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.947369 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e42fb-07e9-4152-9fc2-01c29b71592b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.956817 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efde9d-487e-4d15-8845-01bcd602ea4d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:43.992891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48777ba-a70f-4837-8daf-4d61b73ec037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:43.993802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48777ba-a70f-4837-8daf-4d61b73ec037 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:43.994085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48777ba-a70f-4837-8daf-4d61b73ec037 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:43.994362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48777ba-a70f-4837-8daf-4d61b73ec037 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:43.995595 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48777ba-a70f-4837-8daf-4d61b73ec037 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:43.997175 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48777ba-a70f-4837-8daf-4d61b73ec037 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:44.003924 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 48/93] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:44.005558 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48777ba-a70f-4837-8daf-4d61b73ec037 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:44.019589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e42fb-07e9-4152-9fc2-01c29b71592b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:44.021243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e42fb-07e9-4152-9fc2-01c29b71592b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:44.131277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48777ba-a70f-4837-8daf-4d61b73ec037 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:44.144935 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 46/94] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:44.150540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e42fb-07e9-4152-9fc2-01c29b71592b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:44.180017 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:44.181403 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:44.184425 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:44.184425 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:44.184425 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] RBAC: Adding query filter params (role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=8f180df64cf54d3996588ba6518e4e4c, user.id=6a6136a2abe6402cab688d9bb754142c) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:44.184425 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:44.187149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:44.195455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 49/95] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:44 2022] GET /identity/v3/role_assignments?user.id=6a6136a2abe6402cab688d9bb754142c&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:44.269893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e42fb-07e9-4152-9fc2-01c29b71592b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:44.314117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64ae6f-fd36-4c13-9285-22c146518218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:44.315329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ae6f-fd36-4c13-9285-22c146518218 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:44.315873 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ae6f-fd36-4c13-9285-22c146518218 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:44.316374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ae6f-fd36-4c13-9285-22c146518218 None admin] PATH_INFO: `/v3/projects/5208379a2a06419086402c931f47558e` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:44.321781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64ae6f-fd36-4c13-9285-22c146518218 None admin] RBAC: Authorizing `identity:get_project(project_id=5208379a2a06419086402c931f47558e)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:44.323513 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64ae6f-fd36-4c13-9285-22c146518218 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:44.325687 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 47/96] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:44 2022] GET /identity/v3/projects/5208379a2a06419086402c931f47558e => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:44.331865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:44.443204 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f804c08-c380-488d-b9ef-f7557ad11e4f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:44.489948 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e8471-5312-44e2-a19c-2cd3b568f66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:44.490716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8471-5312-44e2-a19c-2cd3b568f66c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:44.490883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8471-5312-44e2-a19c-2cd3b568f66c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:44.491160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8471-5312-44e2-a19c-2cd3b568f66c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:44.492022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8471-5312-44e2-a19c-2cd3b568f66c None admin] RBAC: Adding query filter params (role.id=67832478d7b8428d996975409814f845, scope.project.id=5208379a2a06419086402c931f47558e, user.id=6a6136a2abe6402cab688d9bb754142c) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:44.492338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8471-5312-44e2-a19c-2cd3b568f66c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:44.493872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8471-5312-44e2-a19c-2cd3b568f66c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:44.501568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 50/97] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:44 2022] GET /identity/v3/role_assignments?user.id=6a6136a2abe6402cab688d9bb754142c&scope.project.id=5208379a2a06419086402c931f47558e&role.id=67832478d7b8428d996975409814f845 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:45.780849 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555378d-9f0d-400d-a197-d83329c1de04 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:45.781378 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555378d-9f0d-400d-a197-d83329c1de04 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:45.781452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555378d-9f0d-400d-a197-d83329c1de04 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:45.783065 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 48/98] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:45.788366 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e696d0-5420-4b3a-a535-1ffa71addfec None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:45.788775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e696d0-5420-4b3a-a535-1ffa71addfec None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:45.788937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e696d0-5420-4b3a-a535-1ffa71addfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:45.814098 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-04e696d0-5420-4b3a-a535-1ffa71addfec None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:45.815333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-04e696d0-5420-4b3a-a535-1ffa71addfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:45.899316 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 51/99] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:45.942874 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555378d-9f0d-400d-a197-d83329c1de04 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:46.062744 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555378d-9f0d-400d-a197-d83329c1de04 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:46.106016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13bfcc-1be4-4d10-95ee-8fa49211bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:46.106853 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13bfcc-1be4-4d10-95ee-8fa49211bf7f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:46.107190 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13bfcc-1be4-4d10-95ee-8fa49211bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:46.108032 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13bfcc-1be4-4d10-95ee-8fa49211bf7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:46.130734 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13bfcc-1be4-4d10-95ee-8fa49211bf7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-4f13bfcc-1be4-4d10-95ee-8fa49211bf7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:46.142606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:46.143481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:46.144247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 49/100] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:46.149656 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e696d0-5420-4b3a-a535-1ffa71addfec None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:46.248251 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e696d0-5420-4b3a-a535-1ffa71addfec None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:46.288142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:46.289925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:46.290209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:46.290486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:46.291249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:46.291556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:46.292930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:46.307599 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 52/101] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:46.313908 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13bfcc-1be4-4d10-95ee-8fa49211bf7f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:46.443297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13bfcc-1be4-4d10-95ee-8fa49211bf7f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:46.480429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ef0-cb8a-489d-9946-e300c98d60d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:46.481318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ef0-cb8a-489d-9946-e300c98d60d8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:46.481604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ef0-cb8a-489d-9946-e300c98d60d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:46.481875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc810ef0-cb8a-489d-9946-e300c98d60d8 None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:46.483165 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc810ef0-cb8a-489d-9946-e300c98d60d8 None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:46.484683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc810ef0-cb8a-489d-9946-e300c98d60d8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:46.486485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 50/102] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:46 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:46.491785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:46.656783 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f201e3-81c9-484e-a17e-4d24ab19128e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:46.701337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1fe32-1d8f-4584-8f0c-0cca2b725cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:46.702566 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1fe32-1d8f-4584-8f0c-0cca2b725cce None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:46.703074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1fe32-1d8f-4584-8f0c-0cca2b725cce None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:46.703548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1fe32-1d8f-4584-8f0c-0cca2b725cce None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:46.704939 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1fe32-1d8f-4584-8f0c-0cca2b725cce None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:46.706630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1fe32-1d8f-4584-8f0c-0cca2b725cce None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:46.708843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 53/103] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:46 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:46.714113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ef0-cb8a-489d-9946-e300c98d60d8 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:46.737241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f421778-ca18-4e6f-ac32-e60b09ff1ee3 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:46.737543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f421778-ca18-4e6f-ac32-e60b09ff1ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:46.737807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f421778-ca18-4e6f-ac32-e60b09ff1ee3 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:46.739184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 54/104] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:46.744116 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b831bd-718a-4bf3-aafe-b60ded0a819e None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:46.744421 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b831bd-718a-4bf3-aafe-b60ded0a819e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:46.744705 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b831bd-718a-4bf3-aafe-b60ded0a819e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:46.771955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-77b831bd-718a-4bf3-aafe-b60ded0a819e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:46.773184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-77b831bd-718a-4bf3-aafe-b60ded0a819e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:46.873244 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc810ef0-cb8a-489d-9946-e300c98d60d8 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:46.907518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 55/105] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:46.913755 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3049470-16db-4985-b379-ea89db277464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:46.914666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049470-16db-4985-b379-ea89db277464 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:46.914953 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049470-16db-4985-b379-ea89db277464 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:46.915233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049470-16db-4985-b379-ea89db277464 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c/users/6a6136a2abe6402cab688d9bb754142c/roles/67832478d7b8428d996975409814f845` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:46.917389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3049470-16db-4985-b379-ea89db277464 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f180df64cf54d3996588ba6518e4e4c, user_id=6a6136a2abe6402cab688d9bb754142c, role_id=67832478d7b8428d996975409814f845)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:46.918851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3049470-16db-4985-b379-ea89db277464 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:46.940962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b831bd-718a-4bf3-aafe-b60ded0a819e None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:46.951870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 51/106] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:01:46 2022] PUT /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c/users/6a6136a2abe6402cab688d9bb754142c/roles/67832478d7b8428d996975409814f845 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:01:47.058665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b831bd-718a-4bf3-aafe-b60ded0a819e None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:47.101311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda13d4-7ab5-4425-ab70-24e0e38ba053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:47.102163 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda13d4-7ab5-4425-ab70-24e0e38ba053 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.102454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda13d4-7ab5-4425-ab70-24e0e38ba053 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.102732 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda13d4-7ab5-4425-ab70-24e0e38ba053 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.116736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda13d4-7ab5-4425-ab70-24e0e38ba053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-3fda13d4-7ab5-4425-ab70-24e0e38ba053 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:47.125398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:47.126356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:01:47.127652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 56/107] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:01:47.132810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3049470-16db-4985-b379-ea89db277464 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:47.262512 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3049470-16db-4985-b379-ea89db277464 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:47.297879 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:47.298575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.298864 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.299211 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.300002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:47.300368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:47.301847 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:47.311703 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 52/108] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:47.317359 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda13d4-7ab5-4425-ab70-24e0e38ba053 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:47.444499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda13d4-7ab5-4425-ab70-24e0e38ba053 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:47.485662 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb45d42-32b1-4876-84a7-877080927e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:47.486548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45d42-32b1-4876-84a7-877080927e2f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.486834 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45d42-32b1-4876-84a7-877080927e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.487116 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb45d42-32b1-4876-84a7-877080927e2f None admin] PATH_INFO: `/v3/users/c980ae99c7444e0390a8a5607832c346` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.494400 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb45d42-32b1-4876-84a7-877080927e2f None admin] RBAC: Authorizing `identity:get_user(user_id=c980ae99c7444e0390a8a5607832c346)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:47.495953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb45d42-32b1-4876-84a7-877080927e2f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:47.498426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 57/109] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:47 2022] GET /identity/v3/users/c980ae99c7444e0390a8a5607832c346 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:47.503920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:47.585576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab539b-9e1f-4302-a8e0-47fd1b53d256 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.585860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab539b-9e1f-4302-a8e0-47fd1b53d256 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.586090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab539b-9e1f-4302-a8e0-47fd1b53d256 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.587723 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 58/110] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:47.601569 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2d72-0653-459a-b1f4-9f1f8f44add2 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.602031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2d72-0653-459a-b1f4-9f1f8f44add2 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.602420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a2d72-0653-459a-b1f4-9f1f8f44add2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.617946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808c846-8768-4149-8a57-f03443f0fda6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:47.626491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-0b5a2d72-0653-459a-b1f4-9f1f8f44add2 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:47.627873 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-0b5a2d72-0653-459a-b1f4-9f1f8f44add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:47.655143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231acaf3-4b09-4521-957d-e19b1a3fb3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:47.656264 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231acaf3-4b09-4521-957d-e19b1a3fb3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.656688 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231acaf3-4b09-4521-957d-e19b1a3fb3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.657140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231acaf3-4b09-4521-957d-e19b1a3fb3d5 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.658507 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231acaf3-4b09-4521-957d-e19b1a3fb3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:47.659967 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231acaf3-4b09-4521-957d-e19b1a3fb3d5 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:47.665071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 53/111] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:47 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:47.667951 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231acaf3-4b09-4521-957d-e19b1a3fb3d5 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:47.709968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 59/112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:47.733120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a2d72-0653-459a-b1f4-9f1f8f44add2 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:47.788312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231acaf3-4b09-4521-957d-e19b1a3fb3d5 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:47.822668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b340bd2-0d11-4866-9878-1d832ab7cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:47.823719 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b340bd2-0d11-4866-9878-1d832ab7cc06 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.823835 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b340bd2-0d11-4866-9878-1d832ab7cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.824107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b340bd2-0d11-4866-9878-1d832ab7cc06 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c/users/c980ae99c7444e0390a8a5607832c346/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.826723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b340bd2-0d11-4866-9878-1d832ab7cc06 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f180df64cf54d3996588ba6518e4e4c, user_id=c980ae99c7444e0390a8a5607832c346, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:47.828230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b340bd2-0d11-4866-9878-1d832ab7cc06 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:47.851701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 54/113] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:01:47 2022] PUT /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c/users/c980ae99c7444e0390a8a5607832c346/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:01:47.852978 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11bab-df74-4036-a701-2f6145795d77 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.857971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11bab-df74-4036-a701-2f6145795d77 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.858081 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11bab-df74-4036-a701-2f6145795d77 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.859815 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 55/114] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:47.864960 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a2d72-0653-459a-b1f4-9f1f8f44add2 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:47.866298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb863e93-bf92-445f-9129-aa2574743bad None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.866594 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb863e93-bf92-445f-9129-aa2574743bad None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.866878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb863e93-bf92-445f-9129-aa2574743bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.902061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-cb863e93-bf92-445f-9129-aa2574743bad None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:47.903479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-cb863e93-bf92-445f-9129-aa2574743bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:47.931598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2ce64-a9ac-4dce-a531-3b7637532a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:47.933217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2ce64-a9ac-4dce-a531-3b7637532a7a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.933682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2ce64-a9ac-4dce-a531-3b7637532a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.934443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2ce64-a9ac-4dce-a531-3b7637532a7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.954661 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2ce64-a9ac-4dce-a531-3b7637532a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-27d2ce64-a9ac-4dce-a531-3b7637532a7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:47.970872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:47.971814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:01:47.972575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:01:47.975064 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 60/115] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:01:47.976658 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889a62-091e-4fc9-8fd4-4c1c2e826cc4 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:47.977150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889a62-091e-4fc9-8fd4-4c1c2e826cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:47.977754 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89889a62-091e-4fc9-8fd4-4c1c2e826cc4 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:47.979639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 61/116] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:47.986079 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889a62-091e-4fc9-8fd4-4c1c2e826cc4 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:48.021788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 56/117] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:48.023224 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af8c43-3eba-482c-8df7-73d83e0e300c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.023513 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af8c43-3eba-482c-8df7-73d83e0e300c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.023790 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af8c43-3eba-482c-8df7-73d83e0e300c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.042015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-d9af8c43-3eba-482c-8df7-73d83e0e300c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:48.049183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-d9af8c43-3eba-482c-8df7-73d83e0e300c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:48.123235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 57/118] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:48.123870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af8c43-3eba-482c-8df7-73d83e0e300c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:48.139290 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89889a62-091e-4fc9-8fd4-4c1c2e826cc4 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:48.190061 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:48.190733 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.191026 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.191359 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.192151 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:48.192387 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:48.197884 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:48.211476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 62/119] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:48.213230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:48.253514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af8c43-3eba-482c-8df7-73d83e0e300c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:48.289239 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f298f91-8c5d-44fe-8fd4-a2f627350ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:48.290452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298f91-8c5d-44fe-8fd4-a2f627350ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.290985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298f91-8c5d-44fe-8fd4-a2f627350ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.291270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298f91-8c5d-44fe-8fd4-a2f627350ae6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.304610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f298f91-8c5d-44fe-8fd4-a2f627350ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-1f298f91-8c5d-44fe-8fd4-a2f627350ae6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:48.315048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:48.315910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 58/120] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:01:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:48.317118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f298f91-8c5d-44fe-8fd4-a2f627350ae6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:48.333841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51382a66-fb21-4661-aebd-ed19d1d9b0ff None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:48.377631 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f024e14-c5c7-4817-a373-47428018e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:48.379301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f024e14-c5c7-4817-a373-47428018e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.379729 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f024e14-c5c7-4817-a373-47428018e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.380197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f024e14-c5c7-4817-a373-47428018e5b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.382483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f024e14-c5c7-4817-a373-47428018e5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:48.384575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f024e14-c5c7-4817-a373-47428018e5b8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:48.387071 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 63/121] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:48.388625 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dee0b-f5a3-4028-9586-f9d7f3ec55ff None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.388912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dee0b-f5a3-4028-9586-f9d7f3ec55ff None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.389220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dee0b-f5a3-4028-9586-f9d7f3ec55ff None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.394570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 64/122] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:48.394570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dee0b-f5a3-4028-9586-f9d7f3ec55ff None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:48.438377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f298f91-8c5d-44fe-8fd4-a2f627350ae6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:48.472784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a6384-0e25-430b-8c3e-8ea7efe73126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:48.474111 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a6384-0e25-430b-8c3e-8ea7efe73126 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.474598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a6384-0e25-430b-8c3e-8ea7efe73126 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.475122 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a6384-0e25-430b-8c3e-8ea7efe73126 None admin] PATH_INFO: `/v3/users/3172aa5eb0a34c7085fc4da5b8063aa9` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.493340 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a6384-0e25-430b-8c3e-8ea7efe73126 None admin] RBAC: Authorizing `identity:get_user(user_id=3172aa5eb0a34c7085fc4da5b8063aa9)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:48.495177 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a6384-0e25-430b-8c3e-8ea7efe73126 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:48.498253 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 59/123] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:48 2022] GET /identity/v3/users/3172aa5eb0a34c7085fc4da5b8063aa9 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:48.499028 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a6384-0e25-430b-8c3e-8ea7efe73126 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:48.514633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dee0b-f5a3-4028-9586-f9d7f3ec55ff None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:48.553355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a431b-0a1b-4263-bdb2-891e1c31e583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:48.554422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a431b-0a1b-4263-bdb2-891e1c31e583 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.554894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a431b-0a1b-4263-bdb2-891e1c31e583 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.555355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a431b-0a1b-4263-bdb2-891e1c31e583 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.556347 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a431b-0a1b-4263-bdb2-891e1c31e583 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:48.556843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a431b-0a1b-4263-bdb2-891e1c31e583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:48.558893 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a431b-0a1b-4263-bdb2-891e1c31e583 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:48.567739 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 65/124] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:01:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:48.569442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b9b67-5a64-4bbd-808e-6be276e38e4b None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.569925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b9b67-5a64-4bbd-808e-6be276e38e4b None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.570386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b9b67-5a64-4bbd-808e-6be276e38e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.587883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-9a6b9b67-5a64-4bbd-808e-6be276e38e4b None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:48.589561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-9a6b9b67-5a64-4bbd-808e-6be276e38e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:48.614989 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a6384-0e25-430b-8c3e-8ea7efe73126 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:48.651396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ea3f7-30bb-4ed8-a00b-8205fbf40a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:48.652402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea3f7-30bb-4ed8-a00b-8205fbf40a3e None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.652941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea3f7-30bb-4ed8-a00b-8205fbf40a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.653476 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea3f7-30bb-4ed8-a00b-8205fbf40a3e None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.654416 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ea3f7-30bb-4ed8-a00b-8205fbf40a3e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:48.658722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ea3f7-30bb-4ed8-a00b-8205fbf40a3e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:48.666178 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 66/125] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:48.673357 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b9b67-5a64-4bbd-808e-6be276e38e4b None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:48.707826 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 60/126] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:01:48 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:48.709448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ea3f7-30bb-4ed8-a00b-8205fbf40a3e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:48.829534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b9b67-5a64-4bbd-808e-6be276e38e4b None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:48.874327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0e9e4-3068-4526-81f9-89739992707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:48.875014 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0e9e4-3068-4526-81f9-89739992707e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.875303 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0e9e4-3068-4526-81f9-89739992707e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.875582 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0e9e4-3068-4526-81f9-89739992707e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.921980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ea3f7-30bb-4ed8-a00b-8205fbf40a3e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:48.960276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0e9e4-3068-4526-81f9-89739992707e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:48.961978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0e9e4-3068-4526-81f9-89739992707e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:48.968336 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562cecc-d2c8-4c8c-b755-e28b26ac8b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:48.969389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562cecc-d2c8-4c8c-b755-e28b26ac8b07 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:48.969672 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562cecc-d2c8-4c8c-b755-e28b26ac8b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:48.969949 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562cecc-d2c8-4c8c-b755-e28b26ac8b07 None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:48.971653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562cecc-d2c8-4c8c-b755-e28b26ac8b07 None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:48.973566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562cecc-d2c8-4c8c-b755-e28b26ac8b07 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:48.976175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 61/127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:48 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:48.977128 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562cecc-d2c8-4c8c-b755-e28b26ac8b07 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:49.095126 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 67/128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:49.095951 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0e9e4-3068-4526-81f9-89739992707e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:49.101889 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562cecc-d2c8-4c8c-b755-e28b26ac8b07 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:49.146639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf05c0-7801-48fe-a934-1895e978c43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:49.147439 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf05c0-7801-48fe-a934-1895e978c43d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:49.147722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf05c0-7801-48fe-a934-1895e978c43d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:49.147992 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf05c0-7801-48fe-a934-1895e978c43d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:49.170515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf05c0-7801-48fe-a934-1895e978c43d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-73bf05c0-7801-48fe-a934-1895e978c43d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:49.189168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:49.190101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:49.190875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:49.191455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 62/129] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:49.192073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf05c0-7801-48fe-a934-1895e978c43d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:49.194896 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0e9e4-3068-4526-81f9-89739992707e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:49.231821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948a683-c1ed-42c3-a09a-0391d3b4f464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:49.233269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a683-c1ed-42c3-a09a-0391d3b4f464 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:49.233556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a683-c1ed-42c3-a09a-0391d3b4f464 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:49.233832 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948a683-c1ed-42c3-a09a-0391d3b4f464 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:49.235050 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948a683-c1ed-42c3-a09a-0391d3b4f464 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:49.236616 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948a683-c1ed-42c3-a09a-0391d3b4f464 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:49.238496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 68/130] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:49 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:49.239262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948a683-c1ed-42c3-a09a-0391d3b4f464 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:49.319423 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf05c0-7801-48fe-a934-1895e978c43d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:49.341504 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948a683-c1ed-42c3-a09a-0391d3b4f464 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:49.362659 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c830c-eec0-4533-a6e7-1e6cadf36a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:49.363856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c830c-eec0-4533-a6e7-1e6cadf36a11 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:49.364354 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c830c-eec0-4533-a6e7-1e6cadf36a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:49.364838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c830c-eec0-4533-a6e7-1e6cadf36a11 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:49.366256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c830c-eec0-4533-a6e7-1e6cadf36a11 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:49.367938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c830c-eec0-4533-a6e7-1e6cadf36a11 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:49.373183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 63/131] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:49 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:49.378875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c830c-eec0-4533-a6e7-1e6cadf36a11 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:49.383903 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:49.385493 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:49.385834 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:49.386116 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:49.389432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:49.389756 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:49.392044 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:49.408356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 69/132] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:49.409724 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:49.544787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946922f-945b-48bf-91a9-097110d5468f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:49.597343 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c830c-eec0-4533-a6e7-1e6cadf36a11 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:49.606256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:49.606917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:49.607239 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:49.607515 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:49.608368 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] RBAC: Adding query filter params (role.id=7d37d7b5e03f435fbba6f4b1ee34cf55, scope.project.id=8f180df64cf54d3996588ba6518e4e4c, user.id=3172aa5eb0a34c7085fc4da5b8063aa9) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:49.608689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:49.610163 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:49.619409 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 70/133] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:49 2022] GET /identity/v3/role_assignments?user.id=3172aa5eb0a34c7085fc4da5b8063aa9&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:49.621700 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:49.662534 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c414d-fbc4-4ef0-a68d-9da143bddf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:49.663437 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c414d-fbc4-4ef0-a68d-9da143bddf7c None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:49.663722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c414d-fbc4-4ef0-a68d-9da143bddf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:49.664002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c414d-fbc4-4ef0-a68d-9da143bddf7c None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c/users/6a6136a2abe6402cab688d9bb754142c/roles/88c855fc5a0a48368fb08bf18f344893` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:49.668245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c414d-fbc4-4ef0-a68d-9da143bddf7c None admin] RBAC: Authorizing `identity:create_grant(project_id=8f180df64cf54d3996588ba6518e4e4c, user_id=6a6136a2abe6402cab688d9bb754142c, role_id=88c855fc5a0a48368fb08bf18f344893)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:49.670680 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c414d-fbc4-4ef0-a68d-9da143bddf7c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:49.693625 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 64/134] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:01:49 2022] PUT /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c/users/6a6136a2abe6402cab688d9bb754142c/roles/88c855fc5a0a48368fb08bf18f344893 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:01:49.771675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd34d6-9e57-4a37-a9c4-377e443c4d45 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:49.808839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9ece-26fc-4a01-b4a7-2f6c29385dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:49.809689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9ece-26fc-4a01-b4a7-2f6c29385dab None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:49.809973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9ece-26fc-4a01-b4a7-2f6c29385dab None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:49.810793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9ece-26fc-4a01-b4a7-2f6c29385dab None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:49.812132 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9ece-26fc-4a01-b4a7-2f6c29385dab None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:49.813816 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9ece-26fc-4a01-b4a7-2f6c29385dab None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:49.815572 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 71/135] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:49 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:49.821534 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c414d-fbc4-4ef0-a68d-9da143bddf7c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:49.931385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c414d-fbc4-4ef0-a68d-9da143bddf7c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:49.967827 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a3719-4969-4c7f-b7ed-247cdb9c8450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:49.968753 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a3719-4969-4c7f-b7ed-247cdb9c8450 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:49.969121 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a3719-4969-4c7f-b7ed-247cdb9c8450 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:49.969408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a3719-4969-4c7f-b7ed-247cdb9c8450 None admin] PATH_INFO: `/v3/projects/5208379a2a06419086402c931f47558e` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:49.970657 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a3719-4969-4c7f-b7ed-247cdb9c8450 None admin] RBAC: Authorizing `identity:get_project(project_id=5208379a2a06419086402c931f47558e)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:49.972142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a3719-4969-4c7f-b7ed-247cdb9c8450 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:49.974333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 65/136] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:49 2022] GET /identity/v3/projects/5208379a2a06419086402c931f47558e => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:49.980174 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9ece-26fc-4a01-b4a7-2f6c29385dab None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:50.118518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9ece-26fc-4a01-b4a7-2f6c29385dab None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:50.154903 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493295bf-3076-4faa-9129-9322b965946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:50.155812 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493295bf-3076-4faa-9129-9322b965946b None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:50.156100 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493295bf-3076-4faa-9129-9322b965946b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:50.156383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493295bf-3076-4faa-9129-9322b965946b None admin] PATH_INFO: `/v3/projects/5208379a2a06419086402c931f47558e/users/6a6136a2abe6402cab688d9bb754142c/roles/67832478d7b8428d996975409814f845` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:50.158425 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493295bf-3076-4faa-9129-9322b965946b None admin] RBAC: Authorizing `identity:create_grant(project_id=5208379a2a06419086402c931f47558e, user_id=6a6136a2abe6402cab688d9bb754142c, role_id=67832478d7b8428d996975409814f845)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:50.159882 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493295bf-3076-4faa-9129-9322b965946b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:50.178534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 72/137] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:01:49 2022] PUT /identity/v3/projects/5208379a2a06419086402c931f47558e/users/6a6136a2abe6402cab688d9bb754142c/roles/67832478d7b8428d996975409814f845 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:01:50.244326 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2feec-a14b-47e1-9803-5e778ed3290d None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:50.244873 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2feec-a14b-47e1-9803-5e778ed3290d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:50.245318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2feec-a14b-47e1-9803-5e778ed3290d None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:50.247169 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 66/138] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:50.253646 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811449a-f9bd-49d5-bce7-3a40cad3e6dc None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:50.254298 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811449a-f9bd-49d5-bce7-3a40cad3e6dc None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:50.254776 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811449a-f9bd-49d5-bce7-3a40cad3e6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:50.274531 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-0811449a-f9bd-49d5-bce7-3a40cad3e6dc None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:50.275744 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-0811449a-f9bd-49d5-bce7-3a40cad3e6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:50.360539 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 73/139] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:50.388314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2feec-a14b-47e1-9803-5e778ed3290d None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:50.535917 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2feec-a14b-47e1-9803-5e778ed3290d None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:50.573601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6742cc9-ca49-43da-91a2-e1a7b9f80e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:50.574407 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6742cc9-ca49-43da-91a2-e1a7b9f80e07 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:50.574697 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6742cc9-ca49-43da-91a2-e1a7b9f80e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:50.574979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6742cc9-ca49-43da-91a2-e1a7b9f80e07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:50.592875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6742cc9-ca49-43da-91a2-e1a7b9f80e07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-c6742cc9-ca49-43da-91a2-e1a7b9f80e07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:50.601058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:50.601986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:50.602740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 67/140] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:50.610338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811449a-f9bd-49d5-bce7-3a40cad3e6dc None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:50.730606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26865-bbd1-4fc6-a01c-3ca2f461545f None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:50.730922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26865-bbd1-4fc6-a01c-3ca2f461545f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:50.731224 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f26865-bbd1-4fc6-a01c-3ca2f461545f None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:50.733241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 68/141] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:50.738435 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e1cf-00a5-4273-861a-8ffccaa19c2d None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:50.738728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e1cf-00a5-4273-861a-8ffccaa19c2d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:50.739012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e1cf-00a5-4273-861a-8ffccaa19c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:50.742090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811449a-f9bd-49d5-bce7-3a40cad3e6dc None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:50.768939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-f2e4e1cf-00a5-4273-861a-8ffccaa19c2d None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:50.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-f2e4e1cf-00a5-4273-861a-8ffccaa19c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:50.781011 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:50.781732 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:50.782018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:50.782352 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:50.783126 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:50.783873 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:50.785307 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:50.795361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 74/142] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:50.800512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:50.904725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 69/143] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:50.935928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e1cf-00a5-4273-861a-8ffccaa19c2d None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:50.949311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc8f83-1f0a-4198-a07f-e1d6723df193 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:51.010363 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2858e8-d7a6-47e5-867a-755af1ccf4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:51.011997 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2858e8-d7a6-47e5-867a-755af1ccf4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.014314 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2858e8-d7a6-47e5-867a-755af1ccf4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.015014 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2858e8-d7a6-47e5-867a-755af1ccf4c0 None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.017748 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2858e8-d7a6-47e5-867a-755af1ccf4c0 None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:51.019568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2858e8-d7a6-47e5-867a-755af1ccf4c0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:51.022486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 75/144] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:50 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:51.031133 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2858e8-d7a6-47e5-867a-755af1ccf4c0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:51.105205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e1cf-00a5-4273-861a-8ffccaa19c2d None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:51.174126 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5075f2-f5b9-4e50-af3c-617fba1cc9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:51.174931 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5075f2-f5b9-4e50-af3c-617fba1cc9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.175208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5075f2-f5b9-4e50-af3c-617fba1cc9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.175480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5075f2-f5b9-4e50-af3c-617fba1cc9c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.196806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5075f2-f5b9-4e50-af3c-617fba1cc9c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-9d5075f2-f5b9-4e50-af3c-617fba1cc9c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:51.213653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:51.214528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:51.215372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:51.222313 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2858e8-d7a6-47e5-867a-755af1ccf4c0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:51.224180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 70/145] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:01:51.242450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5075f2-f5b9-4e50-af3c-617fba1cc9c9 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:51.280399 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010866-d151-4cfc-9254-77f8f8b6be30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:51.280975 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010866-d151-4cfc-9254-77f8f8b6be30 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.281308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010866-d151-4cfc-9254-77f8f8b6be30 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.281586 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010866-d151-4cfc-9254-77f8f8b6be30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.362502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2010866-d151-4cfc-9254-77f8f8b6be30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:51.366756 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2010866-d151-4cfc-9254-77f8f8b6be30 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:51.438809 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5075f2-f5b9-4e50-af3c-617fba1cc9c9 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:51.479823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 76/146] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:51.488193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010866-d151-4cfc-9254-77f8f8b6be30 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:51.494248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:51.495186 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.495671 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.496317 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.497322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:51.497857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:51.499455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:51.508524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 71/147] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:51.520343 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:51.592408 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010866-d151-4cfc-9254-77f8f8b6be30 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:51.627579 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c453d-feb9-4f59-a54b-439f9bcd6983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:51.628438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c453d-feb9-4f59-a54b-439f9bcd6983 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.628718 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c453d-feb9-4f59-a54b-439f9bcd6983 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.628987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c453d-feb9-4f59-a54b-439f9bcd6983 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.631259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c453d-feb9-4f59-a54b-439f9bcd6983 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:51.635445 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c453d-feb9-4f59-a54b-439f9bcd6983 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:51.637179 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 77/148] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:51 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:51.645983 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c453d-feb9-4f59-a54b-439f9bcd6983 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:51.755822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c453d-feb9-4f59-a54b-439f9bcd6983 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:51.760626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfe3b6-bd11-4ce9-9c24-a798ea3d00c8 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:51.788473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:51.790319 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.790319 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.790319 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.790579 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] RBAC: Adding query filter params (role.id=67832478d7b8428d996975409814f845, scope.project.id=8f180df64cf54d3996588ba6518e4e4c, user.id=6a6136a2abe6402cab688d9bb754142c) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:51.790807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:51.792255 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:51.801333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 78/149] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:51 2022] GET /identity/v3/role_assignments?user.id=6a6136a2abe6402cab688d9bb754142c&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=67832478d7b8428d996975409814f845 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:51.814793 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9db27b-ef9f-4c3f-b419-1bed6db56c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:51.815773 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db27b-ef9f-4c3f-b419-1bed6db56c0c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.816059 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db27b-ef9f-4c3f-b419-1bed6db56c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.816336 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db27b-ef9f-4c3f-b419-1bed6db56c0c None admin] PATH_INFO: `/v3/users/c980ae99c7444e0390a8a5607832c346` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.817570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9db27b-ef9f-4c3f-b419-1bed6db56c0c None admin] RBAC: Authorizing `identity:get_user(user_id=c980ae99c7444e0390a8a5607832c346)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:51.819056 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9db27b-ef9f-4c3f-b419-1bed6db56c0c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:51.820843 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 72/150] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:51 2022] GET /identity/v3/users/c980ae99c7444e0390a8a5607832c346 => generated 247 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:51.827514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:51.892729 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7a583-740a-429c-bf2f-6def8f9b1c31 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.893022 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7a583-740a-429c-bf2f-6def8f9b1c31 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.893231 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7a583-740a-429c-bf2f-6def8f9b1c31 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.895218 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 73/151] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:51.900921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af0cc-14e8-4798-a48d-63695e3aa290 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:51.901303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af0cc-14e8-4798-a48d-63695e3aa290 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:51.901615 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af0cc-14e8-4798-a48d-63695e3aa290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:51.922259 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-ee5af0cc-14e8-4798-a48d-63695e3aa290 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:51.924249 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-ee5af0cc-14e8-4798-a48d-63695e3aa290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:51.989379 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0300-9ad2-4867-9ae1-17973a19a4d3 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:52.087774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968b196-212e-4ba4-a4cd-f071f4683789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:52.089346 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968b196-212e-4ba4-a4cd-f071f4683789 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:52.089849 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968b196-212e-4ba4-a4cd-f071f4683789 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:52.090036 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 74/152] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:52.090729 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968b196-212e-4ba4-a4cd-f071f4683789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:52.133222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5af0cc-14e8-4798-a48d-63695e3aa290 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:52.192650 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968b196-212e-4ba4-a4cd-f071f4683789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:52.194391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968b196-212e-4ba4-a4cd-f071f4683789 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:52.313423 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5af0cc-14e8-4798-a48d-63695e3aa290 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:52.349072 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 79/153] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:52.352256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968b196-212e-4ba4-a4cd-f071f4683789 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:52.355201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841dfe6-5ff8-4524-a6e7-bc68d856bee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:52.356012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841dfe6-5ff8-4524-a6e7-bc68d856bee2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:52.356295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841dfe6-5ff8-4524-a6e7-bc68d856bee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:52.356568 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841dfe6-5ff8-4524-a6e7-bc68d856bee2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:52.364386 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841dfe6-5ff8-4524-a6e7-bc68d856bee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:52.365889 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841dfe6-5ff8-4524-a6e7-bc68d856bee2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:52.371403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 75/154] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:52.376668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841dfe6-5ff8-4524-a6e7-bc68d856bee2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:52.538413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968b196-212e-4ba4-a4cd-f071f4683789 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:52.585062 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841dfe6-5ff8-4524-a6e7-bc68d856bee2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:52.592142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b918a8-4fd8-4f61-aebf-cfe4098459d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:52.593419 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b918a8-4fd8-4f61-aebf-cfe4098459d6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:52.593968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b918a8-4fd8-4f61-aebf-cfe4098459d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:52.594432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b918a8-4fd8-4f61-aebf-cfe4098459d6 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:52.596886 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b918a8-4fd8-4f61-aebf-cfe4098459d6 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:52.598568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b918a8-4fd8-4f61-aebf-cfe4098459d6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:52.600501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 80/155] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:52 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:52.606681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b918a8-4fd8-4f61-aebf-cfe4098459d6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:52.664924 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7c2af-f813-4a7d-b738-fff806dec16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:52.666077 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7c2af-f813-4a7d-b738-fff806dec16e None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:52.667001 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7c2af-f813-4a7d-b738-fff806dec16e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:52.668120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7c2af-f813-4a7d-b738-fff806dec16e None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:52.671894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7c2af-f813-4a7d-b738-fff806dec16e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:52.674662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7c2af-f813-4a7d-b738-fff806dec16e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:52.736776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 76/156] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:01:52 2022] POST /identity/v3/users => generated 283 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:52.759352 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b918a8-4fd8-4f61-aebf-cfe4098459d6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:52.806669 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e8329-e241-4998-99d8-0796b2ae1455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:52.807668 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e8329-e241-4998-99d8-0796b2ae1455 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:52.808143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e8329-e241-4998-99d8-0796b2ae1455 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:52.808633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e8329-e241-4998-99d8-0796b2ae1455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:52.809732 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e8329-e241-4998-99d8-0796b2ae1455 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=8f180df64cf54d3996588ba6518e4e4c, user.id=c980ae99c7444e0390a8a5607832c346) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:52.810825 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e8329-e241-4998-99d8-0796b2ae1455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:52.812496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e8329-e241-4998-99d8-0796b2ae1455 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:52.819618 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 81/157] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:52 2022] GET /identity/v3/role_assignments?user.id=c980ae99c7444e0390a8a5607832c346&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:52.928231 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562a35-f95d-423b-a38b-ac35655b138b None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:52.928579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562a35-f95d-423b-a38b-ac35655b138b None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:52.928813 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562a35-f95d-423b-a38b-ac35655b138b None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:52.930303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 77/158] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:52.936757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420b914-0f3e-44fc-8dba-aae621704d78 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:52.937525 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420b914-0f3e-44fc-8dba-aae621704d78 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:52.938020 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420b914-0f3e-44fc-8dba-aae621704d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:52.966867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-a420b914-0f3e-44fc-8dba-aae621704d78 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:52.968450 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-a420b914-0f3e-44fc-8dba-aae621704d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:53.059184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 82/159] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:53.082789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5562a35-f95d-423b-a38b-ac35655b138b None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:53.208881 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5562a35-f95d-423b-a38b-ac35655b138b None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:53.244698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbf3f8-ff3c-4ab6-be15-9120e030f152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:53.245919 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbf3f8-ff3c-4ab6-be15-9120e030f152 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.246399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbf3f8-ff3c-4ab6-be15-9120e030f152 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.246878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbf3f8-ff3c-4ab6-be15-9120e030f152 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.258256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbf3f8-ff3c-4ab6-be15-9120e030f152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a0fbf3f8-ff3c-4ab6-be15-9120e030f152 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:53.266902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:53.267789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:01:53.268539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:53.271251 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 78/160] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:53.277156 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420b914-0f3e-44fc-8dba-aae621704d78 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:53.350299 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b15266-4e70-4060-baad-d3fbbf8362c8 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.350588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b15266-4e70-4060-baad-d3fbbf8362c8 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.350782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b15266-4e70-4060-baad-d3fbbf8362c8 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.352436 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 79/161] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:01:53.357665 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f44da-f0ec-419c-9f2d-2869b18b2a39 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.358014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f44da-f0ec-419c-9f2d-2869b18b2a39 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.358218 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f44da-f0ec-419c-9f2d-2869b18b2a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.377346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-247f44da-f0ec-419c-9f2d-2869b18b2a39 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:53.378535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-247f44da-f0ec-419c-9f2d-2869b18b2a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:53.399571 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420b914-0f3e-44fc-8dba-aae621704d78 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:53.442286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:53.443423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.444046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.444561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.445570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:53.446092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:53.447681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:53.455208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 80/162] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:01:53.457080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 83/163] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:53.462314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f44da-f0ec-419c-9f2d-2869b18b2a39 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:53.484730 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:53.591398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f44da-f0ec-419c-9f2d-2869b18b2a39 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:53.610909 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ea396-357d-4830-b29e-560c29a7c547 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:53.641858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc751939-a743-4eeb-ad39-39b1679d7444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:53.644683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc751939-a743-4eeb-ad39-39b1679d7444 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.646563 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc751939-a743-4eeb-ad39-39b1679d7444 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.646987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc751939-a743-4eeb-ad39-39b1679d7444 None admin] PATH_INFO: `/v3/users/3172aa5eb0a34c7085fc4da5b8063aa9` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.648946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc751939-a743-4eeb-ad39-39b1679d7444 None admin] RBAC: Authorizing `identity:get_user(user_id=3172aa5eb0a34c7085fc4da5b8063aa9)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:53.651656 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc751939-a743-4eeb-ad39-39b1679d7444 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:53.657309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 81/164] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:53 2022] GET /identity/v3/users/3172aa5eb0a34c7085fc4da5b8063aa9 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:53.662889 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc751939-a743-4eeb-ad39-39b1679d7444 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:53.672564 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52ace5-5bed-41d0-8548-7785ab151a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:53.673465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ace5-5bed-41d0-8548-7785ab151a4c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.673746 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ace5-5bed-41d0-8548-7785ab151a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.674073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ace5-5bed-41d0-8548-7785ab151a4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.686738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52ace5-5bed-41d0-8548-7785ab151a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-cd52ace5-5bed-41d0-8548-7785ab151a4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:53.696883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:53.697823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:01:53.698665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 84/165] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:01:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:01:53.705923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52ace5-5bed-41d0-8548-7785ab151a4c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:53.788270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc751939-a743-4eeb-ad39-39b1679d7444 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:53.826669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea121ae-ed26-47ec-b7a5-d61659ab42fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:53.827528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea121ae-ed26-47ec-b7a5-d61659ab42fa None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.827809 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea121ae-ed26-47ec-b7a5-d61659ab42fa None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.828081 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea121ae-ed26-47ec-b7a5-d61659ab42fa None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.829313 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea121ae-ed26-47ec-b7a5-d61659ab42fa None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:53.830780 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea121ae-ed26-47ec-b7a5-d61659ab42fa None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:53.831018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52ace5-5bed-41d0-8548-7785ab151a4c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:53.832497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 82/166] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:53 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:53.833747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a34fa-15c3-43a9-a885-6ec3bcf22738 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.834028 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a34fa-15c3-43a9-a885-6ec3bcf22738 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.834300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a34fa-15c3-43a9-a885-6ec3bcf22738 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.835696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 83/167] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:53.839685 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a34fa-15c3-43a9-a885-6ec3bcf22738 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:53.872182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e58ab4-5a8a-46f8-bf21-3138159a6add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:53.873578 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e58ab4-5a8a-46f8-bf21-3138159a6add None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.873863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e58ab4-5a8a-46f8-bf21-3138159a6add None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.874448 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e58ab4-5a8a-46f8-bf21-3138159a6add None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.875484 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e58ab4-5a8a-46f8-bf21-3138159a6add None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:53.875796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e58ab4-5a8a-46f8-bf21-3138159a6add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:53.877923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e58ab4-5a8a-46f8-bf21-3138159a6add None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:53.889059 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 85/168] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:01:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:53.890397 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e027ed-14af-4174-b390-b01bec249ecb None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:53.890682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e027ed-14af-4174-b390-b01bec249ecb None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:53.890961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e027ed-14af-4174-b390-b01bec249ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:53.907709 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-03e027ed-14af-4174-b390-b01bec249ecb None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:53.908875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-03e027ed-14af-4174-b390-b01bec249ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:53.964339 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a34fa-15c3-43a9-a885-6ec3bcf22738 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:53.979738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 86/169] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:53.980966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e027ed-14af-4174-b390-b01bec249ecb None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:54.018731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba39e8d-f187-44ae-b858-df1c007f6e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:54.019842 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba39e8d-f187-44ae-b858-df1c007f6e59 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:54.020125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba39e8d-f187-44ae-b858-df1c007f6e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:54.020514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba39e8d-f187-44ae-b858-df1c007f6e59 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c/users/3172aa5eb0a34c7085fc4da5b8063aa9/roles/7d37d7b5e03f435fbba6f4b1ee34cf55` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:54.026388 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba39e8d-f187-44ae-b858-df1c007f6e59 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f180df64cf54d3996588ba6518e4e4c, user_id=3172aa5eb0a34c7085fc4da5b8063aa9, role_id=7d37d7b5e03f435fbba6f4b1ee34cf55)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:54.029253 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba39e8d-f187-44ae-b858-df1c007f6e59 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:54.052050 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 84/170] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:01:53 2022] PUT /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c/users/3172aa5eb0a34c7085fc4da5b8063aa9/roles/7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:01:54.052771 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba39e8d-f187-44ae-b858-df1c007f6e59 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:54.181521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e027ed-14af-4174-b390-b01bec249ecb None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:54.214367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba39e8d-f187-44ae-b858-df1c007f6e59 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:54.220091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d62f02-a79a-4c4b-92c5-ad06119ac056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:54.220961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62f02-a79a-4c4b-92c5-ad06119ac056 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:54.221274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62f02-a79a-4c4b-92c5-ad06119ac056 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:54.221545 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62f02-a79a-4c4b-92c5-ad06119ac056 None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:54.223155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d62f02-a79a-4c4b-92c5-ad06119ac056 None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:54.224972 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d62f02-a79a-4c4b-92c5-ad06119ac056 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:54.227476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 87/171] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:53 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:54.233540 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d62f02-a79a-4c4b-92c5-ad06119ac056 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:54.291137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a5fad-aff6-494d-beec-7efba366547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:54.291945 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a5fad-aff6-494d-beec-7efba366547f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:54.292220 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a5fad-aff6-494d-beec-7efba366547f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:54.292485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a5fad-aff6-494d-beec-7efba366547f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:54.307899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a5fad-aff6-494d-beec-7efba366547f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-b26a5fad-aff6-494d-beec-7efba366547f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:54.316787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:54.317660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:54.319655 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 85/172] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:54.324429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a5fad-aff6-494d-beec-7efba366547f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:54.411791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d62f02-a79a-4c4b-92c5-ad06119ac056 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:54.431956 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a5fad-aff6-494d-beec-7efba366547f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:54.467031 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:54.467694 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:54.467972 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:54.468243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:54.468994 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:54.469323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:54.470713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:54.484993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2902904-411b-4f1f-b5e9-4a7c55a5ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:54.485332 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 86/173] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:54.485683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2902904-411b-4f1f-b5e9-4a7c55a5ac88 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:54.485964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2902904-411b-4f1f-b5e9-4a7c55a5ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:54.486241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2902904-411b-4f1f-b5e9-4a7c55a5ac88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:54.486510 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:54.595675 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0d363-6db7-49f4-b522-8e37fd95cdc6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:54.639054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da628519-87ae-4730-99eb-713b4275ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:54.639933 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628519-87ae-4730-99eb-713b4275ea7c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:54.640208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628519-87ae-4730-99eb-713b4275ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:54.640479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628519-87ae-4730-99eb-713b4275ea7c None admin] PATH_INFO: `/v3/users/6a6136a2abe6402cab688d9bb754142c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:54.641722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da628519-87ae-4730-99eb-713b4275ea7c None admin] RBAC: Authorizing `identity:get_user(user_id=6a6136a2abe6402cab688d9bb754142c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:54.643160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da628519-87ae-4730-99eb-713b4275ea7c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:54.648989 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 87/174] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:54 2022] GET /identity/v3/users/6a6136a2abe6402cab688d9bb754142c => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:54.650590 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da628519-87ae-4730-99eb-713b4275ea7c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:54.663572 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2902904-411b-4f1f-b5e9-4a7c55a5ac88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:54.664994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2902904-411b-4f1f-b5e9-4a7c55a5ac88 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:54.795716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 88/175] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:54.802354 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2902904-411b-4f1f-b5e9-4a7c55a5ac88 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:54.895148 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da628519-87ae-4730-99eb-713b4275ea7c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:54.929604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2902904-411b-4f1f-b5e9-4a7c55a5ac88 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:54.967462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e30f4-d8c2-4d3a-b5c8-8c2c672a8db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:54.968362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e30f4-d8c2-4d3a-b5c8-8c2c672a8db9 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:54.968643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e30f4-d8c2-4d3a-b5c8-8c2c672a8db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:54.968923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e30f4-d8c2-4d3a-b5c8-8c2c672a8db9 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:54.970876 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e30f4-d8c2-4d3a-b5c8-8c2c672a8db9 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:54.972380 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e30f4-d8c2-4d3a-b5c8-8c2c672a8db9 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:54.974261 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 89/176] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:54 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:54.981174 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e4254-ca80-43da-9eee-c363ad1e6771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:54.981900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e4254-ca80-43da-9eee-c363ad1e6771 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:54.982192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e4254-ca80-43da-9eee-c363ad1e6771 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:54.982471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e4254-ca80-43da-9eee-c363ad1e6771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:54.984408 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e30f4-d8c2-4d3a-b5c8-8c2c672a8db9 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:55.138737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e4254-ca80-43da-9eee-c363ad1e6771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:55.140216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e4254-ca80-43da-9eee-c363ad1e6771 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:55.166949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e30f4-d8c2-4d3a-b5c8-8c2c672a8db9 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:55.234698 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:55.235279 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:55.235574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:55.235843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:55.236686 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] RBAC: Adding query filter params (role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=8f180df64cf54d3996588ba6518e4e4c, user.id=6a6136a2abe6402cab688d9bb754142c) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:55.237002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:55.238463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:55.249756 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 90/177] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:54 2022] GET /identity/v3/role_assignments?user.id=6a6136a2abe6402cab688d9bb754142c&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:55.328716 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 88/178] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:55.334838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:55.489941 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b1b3f-1666-43aa-9dc4-6be398d01add None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:55.538635 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a57ee-b715-4bda-9c50-743f594bca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:55.539510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a57ee-b715-4bda-9c50-743f594bca59 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:55.539793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a57ee-b715-4bda-9c50-743f594bca59 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:55.540066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a57ee-b715-4bda-9c50-743f594bca59 None admin] PATH_INFO: `/v3/projects/5208379a2a06419086402c931f47558e` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:55.541300 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a57ee-b715-4bda-9c50-743f594bca59 None admin] RBAC: Authorizing `identity:get_project(project_id=5208379a2a06419086402c931f47558e)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:55.542733 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a57ee-b715-4bda-9c50-743f594bca59 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:55.544790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 91/179] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:55 2022] GET /identity/v3/projects/5208379a2a06419086402c931f47558e => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:55.550332 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e4254-ca80-43da-9eee-c363ad1e6771 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:55.674068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e4254-ca80-43da-9eee-c363ad1e6771 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:55.714164 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4e9a0-9202-457f-82ed-109a92ceab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:55.715208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4e9a0-9202-457f-82ed-109a92ceab20 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:55.715694 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4e9a0-9202-457f-82ed-109a92ceab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:55.716191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4e9a0-9202-457f-82ed-109a92ceab20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:55.717252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4e9a0-9202-457f-82ed-109a92ceab20 None admin] RBAC: Adding query filter params (role.id=67832478d7b8428d996975409814f845, scope.project.id=5208379a2a06419086402c931f47558e, user.id=6a6136a2abe6402cab688d9bb754142c) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:55.717759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4e9a0-9202-457f-82ed-109a92ceab20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:55.719404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4e9a0-9202-457f-82ed-109a92ceab20 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:55.725899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 89/180] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:55 2022] GET /identity/v3/role_assignments?user.id=6a6136a2abe6402cab688d9bb754142c&scope.project.id=5208379a2a06419086402c931f47558e&role.id=67832478d7b8428d996975409814f845 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:56.190311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d3bdd-0bf4-4ec0-addf-6e7ce9316d48 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.190592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d3bdd-0bf4-4ec0-addf-6e7ce9316d48 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.191040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d3bdd-0bf4-4ec0-addf-6e7ce9316d48 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.192565 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 92/181] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:56.198269 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696f1f-4330-483a-8364-7ddf0279c318 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.198553 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696f1f-4330-483a-8364-7ddf0279c318 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.198832 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b696f1f-4330-483a-8364-7ddf0279c318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.218235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-9b696f1f-4330-483a-8364-7ddf0279c318 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:56.219420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-9b696f1f-4330-483a-8364-7ddf0279c318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:56.299565 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 90/182] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:56.325265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d3bdd-0bf4-4ec0-addf-6e7ce9316d48 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:56.431264 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d3bdd-0bf4-4ec0-addf-6e7ce9316d48 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:56.467975 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16281320-d306-4063-ac38-c0d375c73c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:56.468774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16281320-d306-4063-ac38-c0d375c73c90 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.469065 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16281320-d306-4063-ac38-c0d375c73c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.469353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16281320-d306-4063-ac38-c0d375c73c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.478642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709296-0c06-4105-af27-08fab20c656a None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.479564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709296-0c06-4105-af27-08fab20c656a None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.479738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16281320-d306-4063-ac38-c0d375c73c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:56.480217 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709296-0c06-4105-af27-08fab20c656a None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.482368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 91/183] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:56.487887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06b70c-bc94-445a-a3c9-8b4a59a96427 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.488442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06b70c-bc94-445a-a3c9-8b4a59a96427 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.488906 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06b70c-bc94-445a-a3c9-8b4a59a96427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-16281320-d306-4063-ac38-c0d375c73c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:56.490235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:56.491648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:01:56.492624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 93/184] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:56.498679 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16281320-d306-4063-ac38-c0d375c73c90 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:56.512367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-0b06b70c-bc94-445a-a3c9-8b4a59a96427 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:56.514019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-0b06b70c-bc94-445a-a3c9-8b4a59a96427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:56.593941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 92/185] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:56.617613 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06b70c-bc94-445a-a3c9-8b4a59a96427 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:56.622859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16281320-d306-4063-ac38-c0d375c73c90 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:56.656863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:56.657560 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.657839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.658113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.658873 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:56.659177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:56.660553 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:56.668995 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 94/186] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:56.678414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:56.723228 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06b70c-bc94-445a-a3c9-8b4a59a96427 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:56.757510 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc942f8b-38c2-465d-b902-a89628bce2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:56.758304 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942f8b-38c2-465d-b902-a89628bce2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.758580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942f8b-38c2-465d-b902-a89628bce2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.758848 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc942f8b-38c2-465d-b902-a89628bce2f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.769021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc942f8b-38c2-465d-b902-a89628bce2f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-cc942f8b-38c2-465d-b902-a89628bce2f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:56.777283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:56.778194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:56.779556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 93/187] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:01:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:01:56.783817 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c9df1-4310-4742-802c-c747f27d9bac None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:56.784805 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc942f8b-38c2-465d-b902-a89628bce2f0 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:56.819545 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a955174-5c7c-48c1-a7b2-3b6ca613a3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:56.820527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a955174-5c7c-48c1-a7b2-3b6ca613a3cb None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.820744 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a955174-5c7c-48c1-a7b2-3b6ca613a3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.821085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a955174-5c7c-48c1-a7b2-3b6ca613a3cb None admin] PATH_INFO: `/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.835960 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a955174-5c7c-48c1-a7b2-3b6ca613a3cb None admin] RBAC: Authorizing `identity:get_user(user_id=1f82e5ebab844a9ab0cd3e5d21866e90)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:56.837455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a955174-5c7c-48c1-a7b2-3b6ca613a3cb None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:56.839551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 95/188] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:56 2022] GET /identity/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:01:56.844966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a955174-5c7c-48c1-a7b2-3b6ca613a3cb None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:56.893199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc942f8b-38c2-465d-b902-a89628bce2f0 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:56.928114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66752d04-4ef2-4b85-b2d5-7488c0283c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:56.929096 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66752d04-4ef2-4b85-b2d5-7488c0283c12 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.929549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66752d04-4ef2-4b85-b2d5-7488c0283c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.929990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66752d04-4ef2-4b85-b2d5-7488c0283c12 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.930921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66752d04-4ef2-4b85-b2d5-7488c0283c12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:56.931409 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66752d04-4ef2-4b85-b2d5-7488c0283c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:56.932949 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66752d04-4ef2-4b85-b2d5-7488c0283c12 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:56.945679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 94/189] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:01:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:56.947160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfb95c-3db8-4251-8aae-0477735ac1d8 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.947583 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfb95c-3db8-4251-8aae-0477735ac1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.947971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a955174-5c7c-48c1-a7b2-3b6ca613a3cb None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:56.948169 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfb95c-3db8-4251-8aae-0477735ac1d8 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:56.950148 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 95/190] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:56.951101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfb95c-3db8-4251-8aae-0477735ac1d8 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:56.985343 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fcd0a-414c-4a0a-8276-65c8b56c361d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:56.985905 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fcd0a-414c-4a0a-8276-65c8b56c361d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:56.986181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fcd0a-414c-4a0a-8276-65c8b56c361d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:56.986447 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fcd0a-414c-4a0a-8276-65c8b56c361d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.050967 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fcd0a-414c-4a0a-8276-65c8b56c361d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.052487 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fcd0a-414c-4a0a-8276-65c8b56c361d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.054003 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfb95c-3db8-4251-8aae-0477735ac1d8 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.088239 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed5e47-9666-406a-b2b1-6ac0bbdfd512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.089128 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5e47-9666-406a-b2b1-6ac0bbdfd512 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.089410 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5e47-9666-406a-b2b1-6ac0bbdfd512 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.089682 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5e47-9666-406a-b2b1-6ac0bbdfd512 None admin] PATH_INFO: `/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.090875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed5e47-9666-406a-b2b1-6ac0bbdfd512 None admin] RBAC: Authorizing `identity:get_user(user_id=1f82e5ebab844a9ab0cd3e5d21866e90)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.092353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed5e47-9666-406a-b2b1-6ac0bbdfd512 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.094029 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 96/191] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:56 2022] GET /identity/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:57.095142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547dff-d98c-4032-803a-2d3a6caac355 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.095428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547dff-d98c-4032-803a-2d3a6caac355 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.095706 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb547dff-d98c-4032-803a-2d3a6caac355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.113735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-eb547dff-d98c-4032-803a-2d3a6caac355 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:57.114200 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-eb547dff-d98c-4032-803a-2d3a6caac355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:57.149805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 96/192] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:57.152175 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe65000-a108-4512-a08f-6a0983b4df3c None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.152463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe65000-a108-4512-a08f-6a0983b4df3c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.152735 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe65000-a108-4512-a08f-6a0983b4df3c None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.155196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 97/193] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:57.155903 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe65000-a108-4512-a08f-6a0983b4df3c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.179280 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 97/194] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:57.181938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb547dff-d98c-4032-803a-2d3a6caac355 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.250426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe65000-a108-4512-a08f-6a0983b4df3c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.276290 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb547dff-d98c-4032-803a-2d3a6caac355 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.283344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9083fc-cbca-45a0-ae9e-d5022a3ed5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.283983 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9083fc-cbca-45a0-ae9e-d5022a3ed5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.284256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9083fc-cbca-45a0-ae9e-d5022a3ed5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.284526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9083fc-cbca-45a0-ae9e-d5022a3ed5b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.308245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eefa7-3a68-49c9-b4bd-56b3807f7968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.309124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eefa7-3a68-49c9-b4bd-56b3807f7968 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.309407 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eefa7-3a68-49c9-b4bd-56b3807f7968 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.309679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eefa7-3a68-49c9-b4bd-56b3807f7968 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.311383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3eefa7-3a68-49c9-b4bd-56b3807f7968 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.312846 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3eefa7-3a68-49c9-b4bd-56b3807f7968 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.314570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 98/195] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:57.315686 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0730-c8e5-4602-a0f3-8209b4f86e14 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.315969 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0730-c8e5-4602-a0f3-8209b4f86e14 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.316245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0730-c8e5-4602-a0f3-8209b4f86e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.331956 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-f96e0730-c8e5-4602-a0f3-8209b4f86e14 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:01:57.333162 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-f96e0730-c8e5-4602-a0f3-8209b4f86e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:01:57.350303 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9083fc-cbca-45a0-ae9e-d5022a3ed5b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.351751 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9083fc-cbca-45a0-ae9e-d5022a3ed5b7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.396519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 99/196] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:01:57.397311 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e0730-c8e5-4602-a0f3-8209b4f86e14 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.444776 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 98/197] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:01:57.445630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9083fc-cbca-45a0-ae9e-d5022a3ed5b7 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.493840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e0730-c8e5-4602-a0f3-8209b4f86e14 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.525603 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c815c-3b59-4c84-9981-b9bb851748eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.526321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c815c-3b59-4c84-9981-b9bb851748eb None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.526603 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c815c-3b59-4c84-9981-b9bb851748eb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.526877 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c815c-3b59-4c84-9981-b9bb851748eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.528025 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c815c-3b59-4c84-9981-b9bb851748eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.530079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c815c-3b59-4c84-9981-b9bb851748eb None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.531680 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 100/198] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:57.532434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c815c-3b59-4c84-9981-b9bb851748eb None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.538548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9083fc-cbca-45a0-ae9e-d5022a3ed5b7 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.571463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.572112 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.572393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.572670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.573532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=a2350e443c204e6d9d190688211b1729, user.id=1f82e5ebab844a9ab0cd3e5d21866e90) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:57.573849 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.575283 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.580703 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 99/199] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/role_assignments?user.id=1f82e5ebab844a9ab0cd3e5d21866e90&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:57.581306 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.629680 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c815c-3b59-4c84-9981-b9bb851748eb None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.665386 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762c7f-224e-46ab-819e-0cd2826b9383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.666189 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762c7f-224e-46ab-819e-0cd2826b9383 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.666470 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762c7f-224e-46ab-819e-0cd2826b9383 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.666745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762c7f-224e-46ab-819e-0cd2826b9383 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.678357 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31762c7f-224e-46ab-819e-0cd2826b9383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-31762c7f-224e-46ab-819e-0cd2826b9383 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:01:57.686341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:01:57.687295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:01:57.688089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 101/200] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:01:57.688746 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762c7f-224e-46ab-819e-0cd2826b9383 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.703736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078714b-ee72-4dea-a80e-1a3fd423ed1e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.737923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd1172-813c-4a0c-a058-c0dcd95771b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.738792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd1172-813c-4a0c-a058-c0dcd95771b0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.739069 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd1172-813c-4a0c-a058-c0dcd95771b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.739341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd1172-813c-4a0c-a058-c0dcd95771b0 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.740566 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd1172-813c-4a0c-a058-c0dcd95771b0 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.742064 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd1172-813c-4a0c-a058-c0dcd95771b0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.748718 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 100/201] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:57.748718 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd1172-813c-4a0c-a058-c0dcd95771b0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.805127 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762c7f-224e-46ab-819e-0cd2826b9383 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.838058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a5152-767f-4ed9-9956-d264a6cd6cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.839002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5152-767f-4ed9-9956-d264a6cd6cfa None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.839280 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5152-767f-4ed9-9956-d264a6cd6cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.839555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a5152-767f-4ed9-9956-d264a6cd6cfa None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.840266 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a5152-767f-4ed9-9956-d264a6cd6cfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.841741 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a5152-767f-4ed9-9956-d264a6cd6cfa None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.844147 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd1172-813c-4a0c-a058-c0dcd95771b0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:57.862437 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 102/202] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:01:57 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:01:57.863560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a5152-767f-4ed9-9956-d264a6cd6cfa None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:57.894566 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:57.895692 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:57.896291 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:57.896795 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:57.897829 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:57.898351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:57.900117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:57.910676 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 101/203] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:57.915702 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:58.003327 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a5152-767f-4ed9-9956-d264a6cd6cfa None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:58.047888 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:58.048552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:58.048810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101af54-d39b-40c6-9a50-3cbe34927481 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:58.048910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:58.049118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:58.049978 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] RBAC: Adding query filter params (role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=a2350e443c204e6d9d190688211b1729, user.id=1f82e5ebab844a9ab0cd3e5d21866e90) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:58.050461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:58.052775 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:58.061859 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 103/204] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/role_assignments?user.id=1f82e5ebab844a9ab0cd3e5d21866e90&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:58.096637 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47145aaf-84eb-44e4-b1ef-e0e8b0dfee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:58.096873 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47145aaf-84eb-44e4-b1ef-e0e8b0dfee76 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:58.097109 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47145aaf-84eb-44e4-b1ef-e0e8b0dfee76 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:58.098062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47145aaf-84eb-44e4-b1ef-e0e8b0dfee76 None admin] PATH_INFO: `/v3/users/3172aa5eb0a34c7085fc4da5b8063aa9` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:58.098681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47145aaf-84eb-44e4-b1ef-e0e8b0dfee76 None admin] RBAC: Authorizing `identity:get_user(user_id=3172aa5eb0a34c7085fc4da5b8063aa9)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:58.100268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47145aaf-84eb-44e4-b1ef-e0e8b0dfee76 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:58.102552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 102/205] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:01:57 2022] GET /identity/v3/users/3172aa5eb0a34c7085fc4da5b8063aa9 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:58.108248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:58.226154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc24d55-572b-485f-8af7-b1797a878082 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:58.267423 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69208802-621e-47a2-895b-8d65a98a19f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:58.268079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69208802-621e-47a2-895b-8d65a98a19f2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:58.268358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69208802-621e-47a2-895b-8d65a98a19f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:58.268630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69208802-621e-47a2-895b-8d65a98a19f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:58.348462 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69208802-621e-47a2-895b-8d65a98a19f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:58.350538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69208802-621e-47a2-895b-8d65a98a19f2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:58.451034 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 104/206] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:01:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:01:58.456797 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47145aaf-84eb-44e4-b1ef-e0e8b0dfee76 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:58.560637 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47145aaf-84eb-44e4-b1ef-e0e8b0dfee76 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:58.604480 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d960a8-3592-4f2f-9412-59e737f2683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:58.605685 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d960a8-3592-4f2f-9412-59e737f2683d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:58.606184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d960a8-3592-4f2f-9412-59e737f2683d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:58.606648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d960a8-3592-4f2f-9412-59e737f2683d None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:58.608152 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d960a8-3592-4f2f-9412-59e737f2683d None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:58.610344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d960a8-3592-4f2f-9412-59e737f2683d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:58.613317 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 103/207] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:01:58 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:58.620200 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69208802-621e-47a2-895b-8d65a98a19f2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:01:58.732982 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69208802-621e-47a2-895b-8d65a98a19f2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:01:58.772457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d36a4f-e5de-4a4c-979b-3b40d8a5c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:01:58.773487 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d36a4f-e5de-4a4c-979b-3b40d8a5c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:58.773962 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d36a4f-e5de-4a4c-979b-3b40d8a5c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:58.774444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d36a4f-e5de-4a4c-979b-3b40d8a5c4d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:58.775484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d36a4f-e5de-4a4c-979b-3b40d8a5c4d4 None admin] RBAC: Adding query filter params (role.id=7d37d7b5e03f435fbba6f4b1ee34cf55, scope.project.id=8f180df64cf54d3996588ba6518e4e4c, user.id=3172aa5eb0a34c7085fc4da5b8063aa9) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:01:58.775986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d36a4f-e5de-4a4c-979b-3b40d8a5c4d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:01:58.777646 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d36a4f-e5de-4a4c-979b-3b40d8a5c4d4 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:01:58.784565 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 105/208] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:01:58 2022] GET /identity/v3/role_assignments?user.id=3172aa5eb0a34c7085fc4da5b8063aa9&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:01:59.972662 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e046-ceb7-4c6c-83de-be2ed932a797 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:59.973203 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e046-ceb7-4c6c-83de-be2ed932a797 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:59.973279 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e1e046-ceb7-4c6c-83de-be2ed932a797 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:59.974596 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 104/209] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:01:59.980183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc426-e950-4d5c-8d5e-128a589c3c51 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:01:59.980950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc426-e950-4d5c-8d5e-128a589c3c51 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:01:59.981384 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc426-e950-4d5c-8d5e-128a589c3c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:01:59.999334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-30ccc426-e950-4d5c-8d5e-128a589c3c51 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:00.000747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-30ccc426-e950-4d5c-8d5e-128a589c3c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:00.067941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 106/210] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:00.089838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1e046-ceb7-4c6c-83de-be2ed932a797 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:00.194421 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e1e046-ceb7-4c6c-83de-be2ed932a797 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:00.228235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5feb49-e8f7-459c-a8c1-4a670f9ccf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:00.228940 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5feb49-e8f7-459c-a8c1-4a670f9ccf7e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.229247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5feb49-e8f7-459c-a8c1-4a670f9ccf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.229520 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5feb49-e8f7-459c-a8c1-4a670f9ccf7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.239850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5feb49-e8f7-459c-a8c1-4a670f9ccf7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-cd5feb49-e8f7-459c-a8c1-4a670f9ccf7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:00.247485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:00.248376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:00.249274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 105/211] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:00.254705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc426-e950-4d5c-8d5e-128a589c3c51 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:00.369187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc426-e950-4d5c-8d5e-128a589c3c51 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:00.405073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639321f-9fe1-455d-9c31-e06076af9586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:00.405733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639321f-9fe1-455d-9c31-e06076af9586 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.406012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639321f-9fe1-455d-9c31-e06076af9586 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.406281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639321f-9fe1-455d-9c31-e06076af9586 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.407028 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639321f-9fe1-455d-9c31-e06076af9586 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:00.407326 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639321f-9fe1-455d-9c31-e06076af9586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:00.408699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639321f-9fe1-455d-9c31-e06076af9586 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:00.416900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 107/212] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:00.423129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5feb49-e8f7-459c-a8c1-4a670f9ccf7e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:00.528432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5feb49-e8f7-459c-a8c1-4a670f9ccf7e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:00.541957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a3ca5-4890-4bd7-a747-8c218bc6f237 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.542254 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a3ca5-4890-4bd7-a747-8c218bc6f237 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.542491 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a3ca5-4890-4bd7-a747-8c218bc6f237 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.544984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 108/213] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:00.548608 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562610df-0a11-4f9f-b3d8-2055de3d396b None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.548884 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562610df-0a11-4f9f-b3d8-2055de3d396b None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.549187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562610df-0a11-4f9f-b3d8-2055de3d396b None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.550668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 109/214] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:00.551796 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3991048-5484-4258-8430-3666e363431e None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.552078 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3991048-5484-4258-8430-3666e363431e None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.552348 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3991048-5484-4258-8430-3666e363431e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.567443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523c2cc-750e-4133-b220-cad3a638046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:00.568327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523c2cc-750e-4133-b220-cad3a638046a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.568606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523c2cc-750e-4133-b220-cad3a638046a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.568880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523c2cc-750e-4133-b220-cad3a638046a None admin] PATH_INFO: `/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.570151 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523c2cc-750e-4133-b220-cad3a638046a None admin] RBAC: Authorizing `identity:get_user(user_id=1f82e5ebab844a9ab0cd3e5d21866e90)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:00.570466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-e3991048-5484-4258-8430-3666e363431e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:00.571619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523c2cc-750e-4133-b220-cad3a638046a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:00.571714 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-e3991048-5484-4258-8430-3666e363431e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:00.573510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 106/215] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:00 2022] GET /identity/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:00.574458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45a80-c82a-4174-ab3a-079207d94412 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.574836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45a80-c82a-4174-ab3a-079207d94412 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.575117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45a80-c82a-4174-ab3a-079207d94412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.591609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-33f45a80-c82a-4174-ab3a-079207d94412 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:00.592844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-33f45a80-c82a-4174-ab3a-079207d94412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:00.644340 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 110/216] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:00.645455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3991048-5484-4258-8430-3666e363431e None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:00.659775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 107/217] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:00.666887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f45a80-c82a-4174-ab3a-079207d94412 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:00.748756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3991048-5484-4258-8430-3666e363431e None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:00.761201 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f45a80-c82a-4174-ab3a-079207d94412 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:00.779818 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9f47b-559d-46bf-a5f1-8a42c3d75ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:00.780598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9f47b-559d-46bf-a5f1-8a42c3d75ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.780874 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9f47b-559d-46bf-a5f1-8a42c3d75ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.781176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9f47b-559d-46bf-a5f1-8a42c3d75ed2 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.782333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9f47b-559d-46bf-a5f1-8a42c3d75ed2 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:00.783779 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9f47b-559d-46bf-a5f1-8a42c3d75ed2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:00.785478 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 111/218] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:00 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:00.787540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17426e8a-df5d-4a05-b92f-140e4c314c97 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.787540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17426e8a-df5d-4a05-b92f-140e4c314c97 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.787540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17426e8a-df5d-4a05-b92f-140e4c314c97 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.788497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 112/219] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:00.788999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17426e8a-df5d-4a05-b92f-140e4c314c97 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:00.794739 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f198b96-f090-4630-b20d-c2b74d66fd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:00.795567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f198b96-f090-4630-b20d-c2b74d66fd80 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.795844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f198b96-f090-4630-b20d-c2b74d66fd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.796120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f198b96-f090-4630-b20d-c2b74d66fd80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.806036 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f198b96-f090-4630-b20d-c2b74d66fd80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-6f198b96-f090-4630-b20d-c2b74d66fd80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:00.814993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:00.815933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:00.816719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 108/220] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:00.818081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f198b96-f090-4630-b20d-c2b74d66fd80 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:00.893159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17426e8a-df5d-4a05-b92f-140e4c314c97 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:00.914621 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f198b96-f090-4630-b20d-c2b74d66fd80 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:00.924710 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03be23-ecc5-4b03-b5a0-57441a9514bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:00.925520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03be23-ecc5-4b03-b5a0-57441a9514bc None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.925794 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03be23-ecc5-4b03-b5a0-57441a9514bc None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.926062 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03be23-ecc5-4b03-b5a0-57441a9514bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.927194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03be23-ecc5-4b03-b5a0-57441a9514bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:00.928630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03be23-ecc5-4b03-b5a0-57441a9514bc None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:00.930295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 113/221] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:00.931399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb351d7-1841-44cb-a6d9-738565d3c57c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.931677 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb351d7-1841-44cb-a6d9-738565d3c57c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.931947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb351d7-1841-44cb-a6d9-738565d3c57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.947747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-abb351d7-1841-44cb-a6d9-738565d3c57c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:00.948606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bdbef-a4bc-4220-a861-4575cc277c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:00.948955 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-abb351d7-1841-44cb-a6d9-738565d3c57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:00.949531 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bdbef-a4bc-4220-a861-4575cc277c95 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:00.949809 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bdbef-a4bc-4220-a861-4575cc277c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:00.950080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bdbef-a4bc-4220-a861-4575cc277c95 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729/users/1f82e5ebab844a9ab0cd3e5d21866e90/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:00.952027 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bdbef-a4bc-4220-a861-4575cc277c95 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2350e443c204e6d9d190688211b1729, user_id=1f82e5ebab844a9ab0cd3e5d21866e90, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:00.953484 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bdbef-a4bc-4220-a861-4575cc277c95 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:00.965367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 109/222] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:00 2022] PUT /identity/v3/projects/a2350e443c204e6d9d190688211b1729/users/1f82e5ebab844a9ab0cd3e5d21866e90/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:00.966141 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bdbef-a4bc-4220-a861-4575cc277c95 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.023745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 114/223] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:01.034246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb351d7-1841-44cb-a6d9-738565d3c57c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.076901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bdbef-a4bc-4220-a861-4575cc277c95 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.116541 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.117081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.117371 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.117644 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.118425 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:01.118732 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:01.120176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:01.130003 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 110/224] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:01.130003 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.143932 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb351d7-1841-44cb-a6d9-738565d3c57c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.177711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fc6a-0326-44d3-b053-e41abed86feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.178388 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fc6a-0326-44d3-b053-e41abed86feb None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.178667 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fc6a-0326-44d3-b053-e41abed86feb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.178936 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fc6a-0326-44d3-b053-e41abed86feb None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.179645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2fc6a-0326-44d3-b053-e41abed86feb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:01.181171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2fc6a-0326-44d3-b053-e41abed86feb None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:01.202108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 115/225] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:01 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:01.202687 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fc6a-0326-44d3-b053-e41abed86feb None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.241907 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a482d81-0767-4b2e-9c01-8c7366726a47 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.283720 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9b9e-1b81-4785-9d6b-641efd4f929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.284971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9b9e-1b81-4785-9d6b-641efd4f929c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.285596 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9b9e-1b81-4785-9d6b-641efd4f929c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.286066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9b9e-1b81-4785-9d6b-641efd4f929c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.298648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc9b9e-1b81-4785-9d6b-641efd4f929c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-53bc9b9e-1b81-4785-9d6b-641efd4f929c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:01.307301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:01.308260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:01.309103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:01.310909 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fc6a-0326-44d3-b053-e41abed86feb None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.312837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 111/226] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:01.321200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9b9e-1b81-4785-9d6b-641efd4f929c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.346085 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced32f2-a178-4500-9e60-ff44a714d2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.346973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced32f2-a178-4500-9e60-ff44a714d2ed None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.347256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced32f2-a178-4500-9e60-ff44a714d2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.347533 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced32f2-a178-4500-9e60-ff44a714d2ed None admin] PATH_INFO: `/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.348802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced32f2-a178-4500-9e60-ff44a714d2ed None admin] RBAC: Authorizing `identity:get_user(user_id=1f82e5ebab844a9ab0cd3e5d21866e90)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:01.350323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced32f2-a178-4500-9e60-ff44a714d2ed None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:01.352344 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 116/227] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:01 2022] GET /identity/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:01.358070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced32f2-a178-4500-9e60-ff44a714d2ed None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.439777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9b9e-1b81-4785-9d6b-641efd4f929c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.466545 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced32f2-a178-4500-9e60-ff44a714d2ed None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.477423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.478209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.478563 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.478843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.479636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:01.479925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:01.481369 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:01.491104 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 112/228] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:01.496839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.509098 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2a0bc-7c66-4ca7-8a0e-663229f91a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.510297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2a0bc-7c66-4ca7-8a0e-663229f91a4f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.510722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2a0bc-7c66-4ca7-8a0e-663229f91a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.511220 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2a0bc-7c66-4ca7-8a0e-663229f91a4f None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.512577 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2a0bc-7c66-4ca7-8a0e-663229f91a4f None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:01.514922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2a0bc-7c66-4ca7-8a0e-663229f91a4f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:01.517912 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 117/229] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:01 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:01.523642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2a0bc-7c66-4ca7-8a0e-663229f91a4f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.615943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8b1e-66fd-4ebd-98de-048d44095945 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.649200 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2a0bc-7c66-4ca7-8a0e-663229f91a4f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.651092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec931451-4ddc-4f0a-b814-c047cbbb3cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.652147 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec931451-4ddc-4f0a-b814-c047cbbb3cce None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.652572 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec931451-4ddc-4f0a-b814-c047cbbb3cce None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.652986 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec931451-4ddc-4f0a-b814-c047cbbb3cce None admin] PATH_INFO: `/v3/users/b0523656cdb9435085d668c24454f937` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.669270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec931451-4ddc-4f0a-b814-c047cbbb3cce None admin] RBAC: Authorizing `identity:get_user(user_id=b0523656cdb9435085d668c24454f937)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:01.670983 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec931451-4ddc-4f0a-b814-c047cbbb3cce None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:01.673181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 113/230] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:01 2022] GET /identity/v3/users/b0523656cdb9435085d668c24454f937 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:01.678658 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec931451-4ddc-4f0a-b814-c047cbbb3cce None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:01.695740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad7b8e-b9f9-45c4-87f8-9a5f5ae98d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.696971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad7b8e-b9f9-45c4-87f8-9a5f5ae98d72 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.697290 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad7b8e-b9f9-45c4-87f8-9a5f5ae98d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.697565 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad7b8e-b9f9-45c4-87f8-9a5f5ae98d72 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729/users/1f82e5ebab844a9ab0cd3e5d21866e90/roles/88c855fc5a0a48368fb08bf18f344893` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.700146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad7b8e-b9f9-45c4-87f8-9a5f5ae98d72 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2350e443c204e6d9d190688211b1729, user_id=1f82e5ebab844a9ab0cd3e5d21866e90, role_id=88c855fc5a0a48368fb08bf18f344893)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:01.702649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad7b8e-b9f9-45c4-87f8-9a5f5ae98d72 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:01.725136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 118/231] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:01 2022] PUT /identity/v3/projects/a2350e443c204e6d9d190688211b1729/users/1f82e5ebab844a9ab0cd3e5d21866e90/roles/88c855fc5a0a48368fb08bf18f344893 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:02:01.794820 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec931451-4ddc-4f0a-b814-c047cbbb3cce None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:01.835547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad892b61-883d-4f61-bf67-e576c6b032d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:01.836521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad892b61-883d-4f61-bf67-e576c6b032d4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:01.836976 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad892b61-883d-4f61-bf67-e576c6b032d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:01.837449 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad892b61-883d-4f61-bf67-e576c6b032d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:01.921895 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad892b61-883d-4f61-bf67-e576c6b032d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:01.923627 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad892b61-883d-4f61-bf67-e576c6b032d4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:02.030863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 114/232] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:02.037155 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad7b8e-b9f9-45c4-87f8-9a5f5ae98d72 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:02.164039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad7b8e-b9f9-45c4-87f8-9a5f5ae98d72 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:02.201212 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a020f-2559-4be5-8f0c-e00ce278aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:02.202003 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a020f-2559-4be5-8f0c-e00ce278aa2f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:02.202282 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a020f-2559-4be5-8f0c-e00ce278aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:02.202557 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a020f-2559-4be5-8f0c-e00ce278aa2f None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:02.203759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a020f-2559-4be5-8f0c-e00ce278aa2f None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:02.205242 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a020f-2559-4be5-8f0c-e00ce278aa2f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:02.211748 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 119/233] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:02 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:02.213099 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad892b61-883d-4f61-bf67-e576c6b032d4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:02.367294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad892b61-883d-4f61-bf67-e576c6b032d4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:02.403265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3bd2c-2b79-4dd1-9d2a-3e592c0103cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:02.404728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3bd2c-2b79-4dd1-9d2a-3e592c0103cf None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:02.405009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3bd2c-2b79-4dd1-9d2a-3e592c0103cf None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:02.405337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3bd2c-2b79-4dd1-9d2a-3e592c0103cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:02.406172 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3bd2c-2b79-4dd1-9d2a-3e592c0103cf None admin] RBAC: Adding query filter params (role.id=67832478d7b8428d996975409814f845, scope.project.id=a2350e443c204e6d9d190688211b1729, user.id=b0523656cdb9435085d668c24454f937) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:02.406478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3bd2c-2b79-4dd1-9d2a-3e592c0103cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:02.407906 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3bd2c-2b79-4dd1-9d2a-3e592c0103cf None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:02.416476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 115/234] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:02 2022] GET /identity/v3/role_assignments?user.id=b0523656cdb9435085d668c24454f937&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=67832478d7b8428d996975409814f845 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:03.544556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686116c-f530-4722-a17e-67d28ba76f02 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:03.545604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686116c-f530-4722-a17e-67d28ba76f02 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:03.546389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686116c-f530-4722-a17e-67d28ba76f02 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:03.548234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 120/235] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:03.559201 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e33b7b-56ee-48b3-98f1-435711d0ee01 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:03.560542 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e33b7b-56ee-48b3-98f1-435711d0ee01 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:03.561019 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e33b7b-56ee-48b3-98f1-435711d0ee01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:03.584836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-15e33b7b-56ee-48b3-98f1-435711d0ee01 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:03.586332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-15e33b7b-56ee-48b3-98f1-435711d0ee01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:03.674998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 116/236] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:03.712307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686116c-f530-4722-a17e-67d28ba76f02 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:03.849987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686116c-f530-4722-a17e-67d28ba76f02 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:03.894690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0e49c-901f-46f2-855e-82c8a2e46f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:03.895501 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0e49c-901f-46f2-855e-82c8a2e46f91 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:03.895787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0e49c-901f-46f2-855e-82c8a2e46f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:03.896087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0e49c-901f-46f2-855e-82c8a2e46f91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:03.917917 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0e49c-901f-46f2-855e-82c8a2e46f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-74a0e49c-901f-46f2-855e-82c8a2e46f91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:03.926142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:03.927067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:03.928268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 121/237] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:03.938149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e33b7b-56ee-48b3-98f1-435711d0ee01 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:03.998525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5805536-a344-4dcd-9e02-9c59cf442dfe None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:03.999034 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5805536-a344-4dcd-9e02-9c59cf442dfe None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:03.999522 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5805536-a344-4dcd-9e02-9c59cf442dfe None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.001864 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 122/238] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:04.009333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03bd40-0369-48ac-99bf-f6a2ad88f2fd None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.009620 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03bd40-0369-48ac-99bf-f6a2ad88f2fd None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.009900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03bd40-0369-48ac-99bf-f6a2ad88f2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.050646 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-7f03bd40-0369-48ac-99bf-f6a2ad88f2fd None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:04.051842 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-7f03bd40-0369-48ac-99bf-f6a2ad88f2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:04.128127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e33b7b-56ee-48b3-98f1-435711d0ee01 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:04.177359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 123/239] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:04.183645 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:04.184308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.184584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.184893 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.185676 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:04.185976 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:04.187344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:04.195629 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 117/240] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:04.200960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03bd40-0369-48ac-99bf-f6a2ad88f2fd None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:04.209892 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:04.363801 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03bd40-0369-48ac-99bf-f6a2ad88f2fd None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:04.388684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed35157-431c-4876-ae81-ceebe1116660 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:04.438258 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398909ad-29f2-4aa1-90e5-b0bff6d5f2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:04.439131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398909ad-29f2-4aa1-90e5-b0bff6d5f2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.439406 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398909ad-29f2-4aa1-90e5-b0bff6d5f2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.439676 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398909ad-29f2-4aa1-90e5-b0bff6d5f2a7 None admin] PATH_INFO: `/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.441489 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398909ad-29f2-4aa1-90e5-b0bff6d5f2a7 None admin] RBAC: Authorizing `identity:get_user(user_id=1f82e5ebab844a9ab0cd3e5d21866e90)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:04.442935 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398909ad-29f2-4aa1-90e5-b0bff6d5f2a7 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:04.444779 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 124/241] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:04 2022] GET /identity/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90 => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:04.452993 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398909ad-29f2-4aa1-90e5-b0bff6d5f2a7 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:04.465245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcc333-e615-4f13-939e-cbb5d471b9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:04.466073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcc333-e615-4f13-939e-cbb5d471b9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.466351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcc333-e615-4f13-939e-cbb5d471b9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.466627 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcc333-e615-4f13-939e-cbb5d471b9c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.486435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dcc333-e615-4f13-939e-cbb5d471b9c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-78dcc333-e615-4f13-939e-cbb5d471b9c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:04.497945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:04.498937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:04.499741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 118/242] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:04.508875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcc333-e615-4f13-939e-cbb5d471b9c0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:04.597201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398909ad-29f2-4aa1-90e5-b0bff6d5f2a7 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:04.621546 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dcc333-e615-4f13-939e-cbb5d471b9c0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:04.630812 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de13a3-5525-4f95-aab6-9c095f4a8599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:04.631467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de13a3-5525-4f95-aab6-9c095f4a8599 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.631750 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de13a3-5525-4f95-aab6-9c095f4a8599 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.632023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de13a3-5525-4f95-aab6-9c095f4a8599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.656267 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf350f5-7230-4128-84ce-8cef84cc989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:04.656934 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf350f5-7230-4128-84ce-8cef84cc989f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.657244 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf350f5-7230-4128-84ce-8cef84cc989f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.657521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf350f5-7230-4128-84ce-8cef84cc989f None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.658283 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf350f5-7230-4128-84ce-8cef84cc989f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:04.658591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf350f5-7230-4128-84ce-8cef84cc989f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:04.659971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf350f5-7230-4128-84ce-8cef84cc989f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:04.668422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 119/243] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:04.670087 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4643bbe-4f2a-405f-9210-628f947800b2 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.670570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4643bbe-4f2a-405f-9210-628f947800b2 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.671073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4643bbe-4f2a-405f-9210-628f947800b2 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.672802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 120/244] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:04.674111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942b1e-b33d-45a3-8dc7-2816b0078755 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.674598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942b1e-b33d-45a3-8dc7-2816b0078755 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.675056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942b1e-b33d-45a3-8dc7-2816b0078755 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.676467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 121/245] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:04.677362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76942b1e-b33d-45a3-8dc7-2816b0078755 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:04.702463 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de13a3-5525-4f95-aab6-9c095f4a8599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:04.703953 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de13a3-5525-4f95-aab6-9c095f4a8599 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:04.782134 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76942b1e-b33d-45a3-8dc7-2816b0078755 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:04.803767 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 125/246] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:04.810177 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610d81-83bc-4473-ad10-05c0638c8444 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.810462 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610d81-83bc-4473-ad10-05c0638c8444 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.810740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610d81-83bc-4473-ad10-05c0638c8444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.815221 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e2b37-29fa-4bda-a45b-2db7450ab47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:04.816255 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e2b37-29fa-4bda-a45b-2db7450ab47f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.816532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e2b37-29fa-4bda-a45b-2db7450ab47f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.816803 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e2b37-29fa-4bda-a45b-2db7450ab47f None admin] PATH_INFO: `/v3/users/d196ee856d5a4ea09a0cef9cf24d3aff` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.826936 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-55610d81-83bc-4473-ad10-05c0638c8444 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:04.828107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-55610d81-83bc-4473-ad10-05c0638c8444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:04.830594 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e2b37-29fa-4bda-a45b-2db7450ab47f None admin] RBAC: Authorizing `identity:get_user(user_id=d196ee856d5a4ea09a0cef9cf24d3aff)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:04.832236 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e2b37-29fa-4bda-a45b-2db7450ab47f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:04.834086 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 122/247] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:04 2022] GET /identity/v3/users/d196ee856d5a4ea09a0cef9cf24d3aff => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:04.838927 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698da2f-1951-48be-b0bb-f853f996baa8 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:04.839215 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698da2f-1951-48be-b0bb-f853f996baa8 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:04.839493 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698da2f-1951-48be-b0bb-f853f996baa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:04.855006 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-4698da2f-1951-48be-b0bb-f853f996baa8 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:04.856162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-4698da2f-1951-48be-b0bb-f853f996baa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:04.892544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 126/248] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:04.893439 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55610d81-83bc-4473-ad10-05c0638c8444 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:04.929747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 123/249] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:04.931068 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698da2f-1951-48be-b0bb-f853f996baa8 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.010031 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55610d81-83bc-4473-ad10-05c0638c8444 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.035248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698da2f-1951-48be-b0bb-f853f996baa8 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.043298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4c112-17ea-40b5-9042-de5b801cf568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.044170 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4c112-17ea-40b5-9042-de5b801cf568 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.044505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4c112-17ea-40b5-9042-de5b801cf568 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.044783 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4c112-17ea-40b5-9042-de5b801cf568 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.045972 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4c112-17ea-40b5-9042-de5b801cf568 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.047422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4c112-17ea-40b5-9042-de5b801cf568 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.053814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 127/250] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:04 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:05.053814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4c112-17ea-40b5-9042-de5b801cf568 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.082820 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc9be1-338d-4193-8692-dbd571781fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.083471 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc9be1-338d-4193-8692-dbd571781fec None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.083751 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc9be1-338d-4193-8692-dbd571781fec None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.084023 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc9be1-338d-4193-8692-dbd571781fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.151931 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4c112-17ea-40b5-9042-de5b801cf568 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.158679 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc9be1-338d-4193-8692-dbd571781fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.160247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc9be1-338d-4193-8692-dbd571781fec None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.184468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5c52f-2690-4850-8a15-1886f69e101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.186118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5c52f-2690-4850-8a15-1886f69e101c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.186437 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5c52f-2690-4850-8a15-1886f69e101c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.186781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5c52f-2690-4850-8a15-1886f69e101c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.196563 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5c52f-2690-4850-8a15-1886f69e101c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-12f5c52f-2690-4850-8a15-1886f69e101c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:05.204584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:05.205610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:05.206403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 128/251] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:05.207097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5c52f-2690-4850-8a15-1886f69e101c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.258268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 124/252] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:05.258268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc9be1-338d-4193-8692-dbd571781fec None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.299599 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5c52f-2690-4850-8a15-1886f69e101c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.330798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40b1b6-3e06-4186-971e-a619d463b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.331571 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40b1b6-3e06-4186-971e-a619d463b33f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.331850 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40b1b6-3e06-4186-971e-a619d463b33f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.332124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40b1b6-3e06-4186-971e-a619d463b33f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.333303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40b1b6-3e06-4186-971e-a619d463b33f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.334752 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40b1b6-3e06-4186-971e-a619d463b33f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.336355 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 129/253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:05.337517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2327f-6e5b-498d-9b5c-3e0f745670b4 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.337800 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2327f-6e5b-498d-9b5c-3e0f745670b4 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.338078 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2327f-6e5b-498d-9b5c-3e0f745670b4 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.339440 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 130/254] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:05.339965 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2327f-6e5b-498d-9b5c-3e0f745670b4 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.346603 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc9be1-338d-4193-8692-dbd571781fec None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.379062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.379708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.380243 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.380521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.381388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=a2350e443c204e6d9d190688211b1729, user.id=1f82e5ebab844a9ab0cd3e5d21866e90) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:05.381700 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.383136 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.388740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 125/255] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/role_assignments?user.id=1f82e5ebab844a9ab0cd3e5d21866e90&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:05.389524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.441109 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2327f-6e5b-498d-9b5c-3e0f745670b4 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.477682 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.478476 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.478832 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.479184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.480038 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:05.480418 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.482005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.490815 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 131/256] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:05.491659 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.493730 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f31e4-e99e-4091-95b9-c8c69c372c95 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.529576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63607eae-4359-4999-87ca-a131758543ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.530461 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63607eae-4359-4999-87ca-a131758543ba None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.530740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63607eae-4359-4999-87ca-a131758543ba None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.531011 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63607eae-4359-4999-87ca-a131758543ba None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.532296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63607eae-4359-4999-87ca-a131758543ba None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.533832 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63607eae-4359-4999-87ca-a131758543ba None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.535607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 126/257] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:05.536825 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aeacb-cf51-4bd5-a15b-363b04bd265b None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.537129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aeacb-cf51-4bd5-a15b-363b04bd265b None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.537406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aeacb-cf51-4bd5-a15b-363b04bd265b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.556010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-ab5aeacb-cf51-4bd5-a15b-363b04bd265b None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:05.557253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-ab5aeacb-cf51-4bd5-a15b-363b04bd265b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:05.595195 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d602d1-a2ad-4e04-a911-e7df81c3b838 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.625634 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 127/258] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:05.633665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aeacb-cf51-4bd5-a15b-363b04bd265b None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.633742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db690a-6980-404f-a7e3-b819248bab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.633742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db690a-6980-404f-a7e3-b819248bab1d None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.633742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db690a-6980-404f-a7e3-b819248bab1d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.633742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db690a-6980-404f-a7e3-b819248bab1d None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.634277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db690a-6980-404f-a7e3-b819248bab1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.634277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db690a-6980-404f-a7e3-b819248bab1d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.655877 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 132/259] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:05 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:05.656722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db690a-6980-404f-a7e3-b819248bab1d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.734658 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aeacb-cf51-4bd5-a15b-363b04bd265b None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.764419 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db690a-6980-404f-a7e3-b819248bab1d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.772451 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53461d-809c-437e-91d4-7641939e4c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.773446 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53461d-809c-437e-91d4-7641939e4c57 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.773724 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53461d-809c-437e-91d4-7641939e4c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.773991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53461d-809c-437e-91d4-7641939e4c57 None admin] PATH_INFO: `/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.775237 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53461d-809c-437e-91d4-7641939e4c57 None admin] RBAC: Authorizing `identity:get_user(user_id=1f82e5ebab844a9ab0cd3e5d21866e90)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.776722 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe53461d-809c-437e-91d4-7641939e4c57 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.783788 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 128/260] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/users/1f82e5ebab844a9ab0cd3e5d21866e90 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:05.783788 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53461d-809c-437e-91d4-7641939e4c57 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.804565 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.805269 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.805550 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.805831 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.806690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] RBAC: Adding query filter params (role.id=7d37d7b5e03f435fbba6f4b1ee34cf55, scope.project.id=a2350e443c204e6d9d190688211b1729, user.id=d196ee856d5a4ea09a0cef9cf24d3aff) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:05.807006 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:05.808486 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:05.815248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 133/261] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/role_assignments?user.id=d196ee856d5a4ea09a0cef9cf24d3aff&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:05.816103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:05.913717 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53461d-809c-437e-91d4-7641939e4c57 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.952469 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c2c29-c908-4b47-8f32-7b33f08392c7 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:05.956138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d32c0-6730-46ff-8bfb-0170c918da09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:05.956959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d32c0-6730-46ff-8bfb-0170c918da09 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:05.957286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d32c0-6730-46ff-8bfb-0170c918da09 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:05.957565 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d32c0-6730-46ff-8bfb-0170c918da09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:05.973075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d32c0-6730-46ff-8bfb-0170c918da09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-bd5d32c0-6730-46ff-8bfb-0170c918da09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:05.982613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:05.983602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:05.984462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 129/262] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:05.991088 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d32c0-6730-46ff-8bfb-0170c918da09 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:06.005429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb11e9-beb3-4aa1-bb79-a82378d30760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:06.007645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb11e9-beb3-4aa1-bb79-a82378d30760 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:06.007925 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb11e9-beb3-4aa1-bb79-a82378d30760 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:06.008199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb11e9-beb3-4aa1-bb79-a82378d30760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:06.097003 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb11e9-beb3-4aa1-bb79-a82378d30760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:06.098890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb11e9-beb3-4aa1-bb79-a82378d30760 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:06.132797 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d32c0-6730-46ff-8bfb-0170c918da09 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:06.171007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:06.171678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:06.171960 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:06.172238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:06.173191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:06.173501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:06.175593 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:06.195402 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 130/263] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:06.201595 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:06.272183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 134/264] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:06.279274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb11e9-beb3-4aa1-bb79-a82378d30760 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:06.384308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f8a50-123a-43fd-851e-a17ab0240f74 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:06.392427 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb11e9-beb3-4aa1-bb79-a82378d30760 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:06.426813 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7fdea-66f9-4905-87cf-cf915f6264ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:06.427672 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7fdea-66f9-4905-87cf-cf915f6264ee None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:06.427951 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7fdea-66f9-4905-87cf-cf915f6264ee None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:06.428223 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7fdea-66f9-4905-87cf-cf915f6264ee None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:06.429441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7fdea-66f9-4905-87cf-cf915f6264ee None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:06.430885 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7fdea-66f9-4905-87cf-cf915f6264ee None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:06.432587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 135/265] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:06 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:06.438826 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7fdea-66f9-4905-87cf-cf915f6264ee None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:06.464469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5adaa-3ff2-43b0-9b73-928cd4d248c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:06.465736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5adaa-3ff2-43b0-9b73-928cd4d248c2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:06.466242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5adaa-3ff2-43b0-9b73-928cd4d248c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:06.466705 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5adaa-3ff2-43b0-9b73-928cd4d248c2 None admin] PATH_INFO: `/v3/users/b0523656cdb9435085d668c24454f937` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:06.469029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5adaa-3ff2-43b0-9b73-928cd4d248c2 None admin] RBAC: Authorizing `identity:get_user(user_id=b0523656cdb9435085d668c24454f937)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:06.470713 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5adaa-3ff2-43b0-9b73-928cd4d248c2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:06.472542 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 131/266] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:06 2022] GET /identity/v3/users/b0523656cdb9435085d668c24454f937 => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:06.478030 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5adaa-3ff2-43b0-9b73-928cd4d248c2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:06.546014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7fdea-66f9-4905-87cf-cf915f6264ee None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:06.586582 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:06.587277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:06.587564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:06.587841 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:06.588683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] RBAC: Adding query filter params (role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=a2350e443c204e6d9d190688211b1729, user.id=1f82e5ebab844a9ab0cd3e5d21866e90) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:06.588998 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:06.590497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:06.603847 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 136/267] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:06 2022] GET /identity/v3/role_assignments?user.id=1f82e5ebab844a9ab0cd3e5d21866e90&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:06.637348 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5adaa-3ff2-43b0-9b73-928cd4d248c2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:06.680848 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd0be9-41d7-4f0f-aa8a-1ae78716c6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:06.681777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd0be9-41d7-4f0f-aa8a-1ae78716c6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:06.682061 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd0be9-41d7-4f0f-aa8a-1ae78716c6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:06.682337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd0be9-41d7-4f0f-aa8a-1ae78716c6f0 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:06.683620 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd0be9-41d7-4f0f-aa8a-1ae78716c6f0 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:06.685771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd0be9-41d7-4f0f-aa8a-1ae78716c6f0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:06.687977 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 132/268] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:06 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:06.693512 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:06.818097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25442d4c-2776-4710-a526-25a9f6c7e2b4 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:06.852801 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced7cd4-e4af-40e7-afe6-46688b57ce03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:06.853731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7cd4-e4af-40e7-afe6-46688b57ce03 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:06.854072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7cd4-e4af-40e7-afe6-46688b57ce03 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:06.854350 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced7cd4-e4af-40e7-afe6-46688b57ce03 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729/users/b0523656cdb9435085d668c24454f937/roles/67832478d7b8428d996975409814f845` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:06.856400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7cd4-e4af-40e7-afe6-46688b57ce03 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2350e443c204e6d9d190688211b1729, user_id=b0523656cdb9435085d668c24454f937, role_id=67832478d7b8428d996975409814f845)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:06.858526 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced7cd4-e4af-40e7-afe6-46688b57ce03 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:06.875326 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 137/269] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:06 2022] PUT /identity/v3/projects/a2350e443c204e6d9d190688211b1729/users/b0523656cdb9435085d668c24454f937/roles/67832478d7b8428d996975409814f845 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:08.214623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f9cb2-0cf5-4170-8b3c-3d674e64792e None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.215620 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f9cb2-0cf5-4170-8b3c-3d674e64792e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.216685 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f9cb2-0cf5-4170-8b3c-3d674e64792e None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.218702 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 133/270] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:08.224851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488dc31-78c4-412c-9103-62c61d0f5067 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.225908 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488dc31-78c4-412c-9103-62c61d0f5067 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.226384 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488dc31-78c4-412c-9103-62c61d0f5067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.243536 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-e488dc31-78c4-412c-9103-62c61d0f5067 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:08.245087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-e488dc31-78c4-412c-9103-62c61d0f5067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:08.318022 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 138/271] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:08.339925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f9cb2-0cf5-4170-8b3c-3d674e64792e None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:08.409477 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05498a-1105-4fe1-999d-b64b2614f984 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.410086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05498a-1105-4fe1-999d-b64b2614f984 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.410578 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05498a-1105-4fe1-999d-b64b2614f984 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.412152 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 139/272] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:02:08.418285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff96db-809f-4b9a-b67b-a6fd0bd9a798 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.418807 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff96db-809f-4b9a-b67b-a6fd0bd9a798 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.419803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff96db-809f-4b9a-b67b-a6fd0bd9a798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.437966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-62ff96db-809f-4b9a-b67b-a6fd0bd9a798 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:08.439434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-62ff96db-809f-4b9a-b67b-a6fd0bd9a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:08.463377 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f9cb2-0cf5-4170-8b3c-3d674e64792e None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:08.502014 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15cc6-c168-4ba9-b021-a43f0b00d70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:08.502806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15cc6-c168-4ba9-b021-a43f0b00d70b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.503086 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15cc6-c168-4ba9-b021-a43f0b00d70b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.503360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15cc6-c168-4ba9-b021-a43f0b00d70b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.505308 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 140/273] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:08.506483 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85eb6-2d2b-4c87-b6a0-dca1aa5af7ce None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.506772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85eb6-2d2b-4c87-b6a0-dca1aa5af7ce None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.507048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead85eb6-2d2b-4c87-b6a0-dca1aa5af7ce None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.508438 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 141/274] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:08.514353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda5a80-eaa1-4f00-843f-7ee6070cee7e None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.514860 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda5a80-eaa1-4f00-843f-7ee6070cee7e None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.515339 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda5a80-eaa1-4f00-843f-7ee6070cee7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.518983 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f15cc6-c168-4ba9-b021-a43f0b00d70b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-c4f15cc6-c168-4ba9-b021-a43f0b00d70b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:08.526782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:08.527673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:08.529807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 134/275] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:08.535109 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15cc6-c168-4ba9-b021-a43f0b00d70b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:08.536915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-ecda5a80-eaa1-4f00-843f-7ee6070cee7e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:08.538122 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-ecda5a80-eaa1-4f00-843f-7ee6070cee7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:08.604666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 142/276] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:08.605349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda5a80-eaa1-4f00-843f-7ee6070cee7e None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:08.639235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15cc6-c168-4ba9-b021-a43f0b00d70b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:08.673840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:08.674430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.674710 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.674985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.675738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:08.676041 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:08.677496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:08.690414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 135/277] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:08.690414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:08.711717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda5a80-eaa1-4f00-843f-7ee6070cee7e None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:08.755148 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c79d2-c337-44b6-a94a-0a13658fb38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:08.755958 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c79d2-c337-44b6-a94a-0a13658fb38f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.756282 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c79d2-c337-44b6-a94a-0a13658fb38f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.756512 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c79d2-c337-44b6-a94a-0a13658fb38f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.768553 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c79d2-c337-44b6-a94a-0a13658fb38f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-485c79d2-c337-44b6-a94a-0a13658fb38f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:08.776312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:08.777450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:08.778297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 143/278] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:08.778974 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c79d2-c337-44b6-a94a-0a13658fb38f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:08.791563 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf9d15-59e1-4c06-9f59-f7aa94e631c8 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:08.823969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb75bf-6a6d-4ff2-8d81-6e5cccb60f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:08.824752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb75bf-6a6d-4ff2-8d81-6e5cccb60f77 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.825082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb75bf-6a6d-4ff2-8d81-6e5cccb60f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.825941 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb75bf-6a6d-4ff2-8d81-6e5cccb60f77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.827186 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb75bf-6a6d-4ff2-8d81-6e5cccb60f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:08.828644 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb75bf-6a6d-4ff2-8d81-6e5cccb60f77 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:08.830459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 136/279] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:08.835532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb75bf-6a6d-4ff2-8d81-6e5cccb60f77 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:08.881537 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c79d2-c337-44b6-a94a-0a13658fb38f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:08.914199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff99c1-9168-4052-b730-946978c9c7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:08.915074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff99c1-9168-4052-b730-946978c9c7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.915355 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff99c1-9168-4052-b730-946978c9c7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.915627 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff99c1-9168-4052-b730-946978c9c7b7 None admin] PATH_INFO: `/v3/users/d196ee856d5a4ea09a0cef9cf24d3aff` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.916831 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff99c1-9168-4052-b730-946978c9c7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=d196ee856d5a4ea09a0cef9cf24d3aff)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:08.918330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff99c1-9168-4052-b730-946978c9c7b7 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:08.925261 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 144/280] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:08 2022] GET /identity/v3/users/d196ee856d5a4ea09a0cef9cf24d3aff => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:08.925261 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff99c1-9168-4052-b730-946978c9c7b7 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:08.934313 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb75bf-6a6d-4ff2-8d81-6e5cccb60f77 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:08.978446 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:08.979120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:08.979398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:08.979671 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:08.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:08.980727 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:08.982685 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:08.997467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 137/281] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:08.997467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:09.037488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff99c1-9168-4052-b730-946978c9c7b7 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:09.072862 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae793446-7bb9-45ee-8050-c960e9e68b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:09.073971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae793446-7bb9-45ee-8050-c960e9e68b0d None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.074489 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae793446-7bb9-45ee-8050-c960e9e68b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.074970 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae793446-7bb9-45ee-8050-c960e9e68b0d None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.076428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae793446-7bb9-45ee-8050-c960e9e68b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:09.078130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae793446-7bb9-45ee-8050-c960e9e68b0d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:09.099729 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 145/282] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:08 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:09.100933 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae793446-7bb9-45ee-8050-c960e9e68b0d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:09.125697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c9e10-c6a8-430c-a826-84663a17815c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:09.164771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdc844-bfda-479c-a3aa-a6c5e34c59a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:09.165693 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdc844-bfda-479c-a3aa-a6c5e34c59a6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.165974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdc844-bfda-479c-a3aa-a6c5e34c59a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.166245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bdc844-bfda-479c-a3aa-a6c5e34c59a6 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.167789 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdc844-bfda-479c-a3aa-a6c5e34c59a6 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:09.169889 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bdc844-bfda-479c-a3aa-a6c5e34c59a6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:09.176910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 138/283] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:08 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:09.177830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdc844-bfda-479c-a3aa-a6c5e34c59a6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:09.267800 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae793446-7bb9-45ee-8050-c960e9e68b0d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:09.291288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bdc844-bfda-479c-a3aa-a6c5e34c59a6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:09.304974 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6846c63-f813-4943-9d8a-7977ddd27cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:09.306441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846c63-f813-4943-9d8a-7977ddd27cba None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.306728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846c63-f813-4943-9d8a-7977ddd27cba None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.307007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846c63-f813-4943-9d8a-7977ddd27cba None admin] PATH_INFO: `/v3/users/d9cce33f079b4fc7ad3e8ceed52c44ac` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.324252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6846c63-f813-4943-9d8a-7977ddd27cba None admin] RBAC: Authorizing `identity:get_user(user_id=d9cce33f079b4fc7ad3e8ceed52c44ac)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:09.326246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6846c63-f813-4943-9d8a-7977ddd27cba None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:09.328876 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 146/284] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:09 2022] GET /identity/v3/users/d9cce33f079b4fc7ad3e8ceed52c44ac => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:09.329880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3142c-7c8b-4931-859f-a800e33be357 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.329880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3142c-7c8b-4931-859f-a800e33be357 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.330043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3142c-7c8b-4931-859f-a800e33be357 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.330697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c57a-fccd-4f35-8675-492230d6d5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:09.331232 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 147/285] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:09.331831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c57a-fccd-4f35-8675-492230d6d5a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.332113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c57a-fccd-4f35-8675-492230d6d5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.332385 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c57a-fccd-4f35-8675-492230d6d5a3 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729/users/d196ee856d5a4ea09a0cef9cf24d3aff/roles/7d37d7b5e03f435fbba6f4b1ee34cf55` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.336706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223c57a-fccd-4f35-8675-492230d6d5a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2350e443c204e6d9d190688211b1729, user_id=d196ee856d5a4ea09a0cef9cf24d3aff, role_id=7d37d7b5e03f435fbba6f4b1ee34cf55)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:09.336879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223c57a-fccd-4f35-8675-492230d6d5a3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:09.338003 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3142c-7c8b-4931-859f-a800e33be357 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:09.364493 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 139/286] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:09 2022] PUT /identity/v3/projects/a2350e443c204e6d9d190688211b1729/users/d196ee856d5a4ea09a0cef9cf24d3aff/roles/7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:09.364493 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd6ef7-e6ae-48e6-ba9e-c5cb93a00096 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.364493 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd6ef7-e6ae-48e6-ba9e-c5cb93a00096 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.364493 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd6ef7-e6ae-48e6-ba9e-c5cb93a00096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.395143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-3cfd6ef7-e6ae-48e6-ba9e-c5cb93a00096 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:09.395143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-3cfd6ef7-e6ae-48e6-ba9e-c5cb93a00096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:09.475570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3142c-7c8b-4931-859f-a800e33be357 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:09.498281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 140/287] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:09.519460 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:09.520126 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.520417 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.520694 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.521587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] RBAC: Adding query filter params (role.id=67832478d7b8428d996975409814f845, user.id=d9cce33f079b4fc7ad3e8ceed52c44ac, scope.system=all) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:09.521904 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:09.523369 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:09.524282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd6ef7-e6ae-48e6-ba9e-c5cb93a00096 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:09.538935 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 148/288] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:02:09 2022] GET /identity/v3/role_assignments?user.id=d9cce33f079b4fc7ad3e8ceed52c44ac&scope.system=all&role.id=67832478d7b8428d996975409814f845 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:09.648087 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd6ef7-e6ae-48e6-ba9e-c5cb93a00096 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:09.685997 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a13722-b094-4d3e-b4db-90c7b164a806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:09.687115 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a13722-b094-4d3e-b4db-90c7b164a806 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.687569 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a13722-b094-4d3e-b4db-90c7b164a806 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.688055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a13722-b094-4d3e-b4db-90c7b164a806 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.699669 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a13722-b094-4d3e-b4db-90c7b164a806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-39a13722-b094-4d3e-b4db-90c7b164a806 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:09.708031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:09.708991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:09.709822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:09.712316 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 141/289] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:09.720782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:09.833473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e350128-d2d5-4832-bb25-904c29cea07e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:09.878968 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:09.879636 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:09.880183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:09.880462 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:09.881241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:09.881549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:09.882925 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:09.896735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 149/290] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:09.902624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a13722-b094-4d3e-b4db-90c7b164a806 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:10.025861 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a13722-b094-4d3e-b4db-90c7b164a806 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:10.074657 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec5d90-a013-42ef-9d14-c2246731c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:10.076108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec5d90-a013-42ef-9d14-c2246731c74b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:10.076387 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec5d90-a013-42ef-9d14-c2246731c74b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:10.076660 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec5d90-a013-42ef-9d14-c2246731c74b None admin] PATH_INFO: `/v3/users/b0523656cdb9435085d668c24454f937` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:10.077924 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec5d90-a013-42ef-9d14-c2246731c74b None admin] RBAC: Authorizing `identity:get_user(user_id=b0523656cdb9435085d668c24454f937)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:10.079388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec5d90-a013-42ef-9d14-c2246731c74b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:10.081204 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 142/291] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:09 2022] GET /identity/v3/users/b0523656cdb9435085d668c24454f937 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:10.086508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:10.230327 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b271bd4-f947-4d98-bff9-d05f1a4225d1 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:10.274987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050edf7-d722-414c-a037-a1d1778244bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:10.275758 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050edf7-d722-414c-a037-a1d1778244bc None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:10.275921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050edf7-d722-414c-a037-a1d1778244bc None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:10.276186 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050edf7-d722-414c-a037-a1d1778244bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:10.348881 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050edf7-d722-414c-a037-a1d1778244bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:10.350308 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050edf7-d722-414c-a037-a1d1778244bc None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:10.449840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 150/292] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:10.458128 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec5d90-a013-42ef-9d14-c2246731c74b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:10.676101 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec5d90-a013-42ef-9d14-c2246731c74b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:10.721674 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104d91e-47dd-4b04-9294-1daf38be25da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:10.722527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104d91e-47dd-4b04-9294-1daf38be25da None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:10.722803 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104d91e-47dd-4b04-9294-1daf38be25da None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:10.723074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104d91e-47dd-4b04-9294-1daf38be25da None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:10.725738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104d91e-47dd-4b04-9294-1daf38be25da None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:10.727176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104d91e-47dd-4b04-9294-1daf38be25da None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:10.729033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 143/293] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:10 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:10.734650 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050edf7-d722-414c-a037-a1d1778244bc None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:10.848653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050edf7-d722-414c-a037-a1d1778244bc None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:10.893734 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a23ace-e1c9-4f18-b0a9-50225e445178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:10.894396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a23ace-e1c9-4f18-b0a9-50225e445178 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:10.894681 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a23ace-e1c9-4f18-b0a9-50225e445178 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:10.894959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a23ace-e1c9-4f18-b0a9-50225e445178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:10.895809 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a23ace-e1c9-4f18-b0a9-50225e445178 None admin] RBAC: Adding query filter params (role.id=67832478d7b8428d996975409814f845, scope.project.id=a2350e443c204e6d9d190688211b1729, user.id=b0523656cdb9435085d668c24454f937) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:10.896120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a23ace-e1c9-4f18-b0a9-50225e445178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:10.897598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a23ace-e1c9-4f18-b0a9-50225e445178 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:10.904093 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 151/294] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:10 2022] GET /identity/v3/role_assignments?user.id=b0523656cdb9435085d668c24454f937&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=67832478d7b8428d996975409814f845 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:11.768290 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c344543-c557-430e-a02e-9e9b50de4aa8 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:11.768850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c344543-c557-430e-a02e-9e9b50de4aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:11.768850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c344543-c557-430e-a02e-9e9b50de4aa8 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:11.770539 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 144/295] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:11.776026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86db54-a098-48e6-a950-99acd7c03602 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:11.776492 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86db54-a098-48e6-a950-99acd7c03602 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:11.776566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86db54-a098-48e6-a950-99acd7c03602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:11.797342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-2e86db54-a098-48e6-a950-99acd7c03602 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:11.798876 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-2e86db54-a098-48e6-a950-99acd7c03602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:11.879180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 152/296] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:11.903544 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c344543-c557-430e-a02e-9e9b50de4aa8 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.010524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c344543-c557-430e-a02e-9e9b50de4aa8 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.045515 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3c2e2-6a60-4ed7-adb5-fb419cf9f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.046697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3c2e2-6a60-4ed7-adb5-fb419cf9f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.047173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3c2e2-6a60-4ed7-adb5-fb419cf9f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.047643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3c2e2-6a60-4ed7-adb5-fb419cf9f7d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.059041 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3c2e2-6a60-4ed7-adb5-fb419cf9f7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-73d3c2e2-6a60-4ed7-adb5-fb419cf9f7d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:12.067604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:12.068588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:12.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:12.072103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 145/297] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:12.080089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86db54-a098-48e6-a950-99acd7c03602 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.090372 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d11-cab0-4587-b83d-a981a5d4576e None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.091259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d11-cab0-4587-b83d-a981a5d4576e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.091497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d11-cab0-4587-b83d-a981a5d4576e None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.093222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 146/298] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:02:12.100085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58572a29-8eac-4bde-bcd9-602bb180cc75 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.100727 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58572a29-8eac-4bde-bcd9-602bb180cc75 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.101257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58572a29-8eac-4bde-bcd9-602bb180cc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.119556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-58572a29-8eac-4bde-bcd9-602bb180cc75 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:12.121129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-58572a29-8eac-4bde-bcd9-602bb180cc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:12.196330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 147/299] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:12.202003 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86db54-a098-48e6-a950-99acd7c03602 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.217209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58572a29-8eac-4bde-bcd9-602bb180cc75 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.236696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.237480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.237664 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.237927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.238769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:12.239077 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:12.240581 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:12.253899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 153/300] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:12.255395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.331173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58572a29-8eac-4bde-bcd9-602bb180cc75 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.352622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb9fc-ddeb-4e54-b483-678bf75c1cf6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.365326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6187f-edfe-40c8-8ee4-7aea4086757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.366113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6187f-edfe-40c8-8ee4-7aea4086757e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.366391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6187f-edfe-40c8-8ee4-7aea4086757e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.366661 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6187f-edfe-40c8-8ee4-7aea4086757e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.367844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6187f-edfe-40c8-8ee4-7aea4086757e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:12.369308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6187f-edfe-40c8-8ee4-7aea4086757e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:12.370950 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 148/301] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:12.374294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707dd8d-8a51-4515-a273-0199fd857743 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.374576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707dd8d-8a51-4515-a273-0199fd857743 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.374852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707dd8d-8a51-4515-a273-0199fd857743 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.376454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 149/302] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:12.377622 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e0ea-9755-4339-92c2-b6c9aa438205 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.377904 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e0ea-9755-4339-92c2-b6c9aa438205 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.378173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525e0ea-9755-4339-92c2-b6c9aa438205 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.379360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 150/303] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:12.380948 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525e0ea-9755-4339-92c2-b6c9aa438205 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.387293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d120b56-6f09-4a67-a28e-193983bac151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.388800 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120b56-6f09-4a67-a28e-193983bac151 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.389140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120b56-6f09-4a67-a28e-193983bac151 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.389421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120b56-6f09-4a67-a28e-193983bac151 None admin] PATH_INFO: `/v3/users/46f9f0cc4880475884563fdce7899679` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.402225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d120b56-6f09-4a67-a28e-193983bac151 None admin] RBAC: Authorizing `identity:get_user(user_id=46f9f0cc4880475884563fdce7899679)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:12.403732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d120b56-6f09-4a67-a28e-193983bac151 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:12.405562 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 154/304] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/users/46f9f0cc4880475884563fdce7899679 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:12.406577 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137f04-dbe3-4709-997b-03826ccca7c3 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.406861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137f04-dbe3-4709-997b-03826ccca7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.407143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83137f04-dbe3-4709-997b-03826ccca7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.422243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-83137f04-dbe3-4709-997b-03826ccca7c3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:12.423411 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-83137f04-dbe3-4709-997b-03826ccca7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:12.476593 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525e0ea-9755-4339-92c2-b6c9aa438205 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.487323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 155/305] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:12.488669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a236cdb-ec95-49e5-9617-2ee702a02744 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.488958 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a236cdb-ec95-49e5-9617-2ee702a02744 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.489411 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a236cdb-ec95-49e5-9617-2ee702a02744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.505514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-5a236cdb-ec95-49e5-9617-2ee702a02744 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:12.506671 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-5a236cdb-ec95-49e5-9617-2ee702a02744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:12.510588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76820-53e6-4033-893d-9a5752c5b23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.511255 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76820-53e6-4033-893d-9a5752c5b23b None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.511533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76820-53e6-4033-893d-9a5752c5b23b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.511808 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76820-53e6-4033-893d-9a5752c5b23b None admin] PATH_INFO: `/v3/groups` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.512526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb76820-53e6-4033-893d-9a5752c5b23b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:12.513983 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb76820-53e6-4033-893d-9a5752c5b23b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:12.527443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 151/306] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 9 15:02:12 2022] POST /identity/v3/groups => generated 229 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:12.528752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76820-53e6-4033-893d-9a5752c5b23b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.577971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 156/307] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:12.579238 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a236cdb-ec95-49e5-9617-2ee702a02744 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.652690 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76820-53e6-4033-893d-9a5752c5b23b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.688202 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.688776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a236cdb-ec95-49e5-9617-2ee702a02744 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.688910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.689169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.689561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.690396 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] RBAC: Adding query filter params (role.id=7d37d7b5e03f435fbba6f4b1ee34cf55, user.id=46f9f0cc4880475884563fdce7899679, scope.system=all) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:12.690714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:12.692183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:12.705093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 152/308] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/role_assignments?user.id=46f9f0cc4880475884563fdce7899679&scope.system=all&role.id=7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:12.705093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.742030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393684-8baa-4d84-a86d-89d49405a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.743208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393684-8baa-4d84-a86d-89d49405a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.743663 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393684-8baa-4d84-a86d-89d49405a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.744079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09393684-8baa-4d84-a86d-89d49405a4f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.758695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09393684-8baa-4d84-a86d-89d49405a4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-09393684-8baa-4d84-a86d-89d49405a4f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:12.768307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:12.769309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:12.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 157/309] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:12.775922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393684-8baa-4d84-a86d-89d49405a4f4 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.826384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83412597-b9f2-4319-b350-ed90f93ba908 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.862265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b643a-edb1-4be6-a503-8d3084bf2248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.863333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b643a-edb1-4be6-a503-8d3084bf2248 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.863791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b643a-edb1-4be6-a503-8d3084bf2248 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.864215 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b643a-edb1-4be6-a503-8d3084bf2248 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.876636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b643a-edb1-4be6-a503-8d3084bf2248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:12.884872 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09393684-8baa-4d84-a86d-89d49405a4f4 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-d38b643a-edb1-4be6-a503-8d3084bf2248 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:12.885870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:12.886953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:12.888398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:12.890710 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 153/310] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:12.898667 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b643a-edb1-4be6-a503-8d3084bf2248 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:12.925789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:12.927249 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:12.927699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:12.928117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:12.929499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] RBAC: Adding query filter params (name=member) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:12.930228 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:12.932223 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:12.943328 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 158/311] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:12.948972 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:13.010175 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b643a-edb1-4be6-a503-8d3084bf2248 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:13.044921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:13.045631 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:13.045910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:13.046183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:13.046935 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:13.047295 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:13.048723 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:13.057429 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 154/312] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:13.067670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:13.079573 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992faf9-c8ad-4562-b162-3d792acff009 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:13.118811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48fc8-643b-41e1-b39c-3400c807b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:13.119516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48fc8-643b-41e1-b39c-3400c807b0fd None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:13.119795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48fc8-643b-41e1-b39c-3400c807b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:13.120127 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48fc8-643b-41e1-b39c-3400c807b0fd None admin] PATH_INFO: `/v3/users/d9cce33f079b4fc7ad3e8ceed52c44ac` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:13.121404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb48fc8-643b-41e1-b39c-3400c807b0fd None admin] RBAC: Authorizing `identity:get_user(user_id=d9cce33f079b4fc7ad3e8ceed52c44ac)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:13.123497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb48fc8-643b-41e1-b39c-3400c807b0fd None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:13.125487 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 159/313] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:12 2022] GET /identity/v3/users/d9cce33f079b4fc7ad3e8ceed52c44ac => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:13.131477 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48fc8-643b-41e1-b39c-3400c807b0fd None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:13.173477 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb73f59-d47d-403e-a938-8e35e087977f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:13.208097 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedde53d-d0c3-4654-88af-562fae334fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:13.208884 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedde53d-d0c3-4654-88af-562fae334fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:13.209186 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedde53d-d0c3-4654-88af-562fae334fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:13.209453 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedde53d-d0c3-4654-88af-562fae334fa7 None admin] PATH_INFO: `/v3/users/d196ee856d5a4ea09a0cef9cf24d3aff` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:13.210651 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedde53d-d0c3-4654-88af-562fae334fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=d196ee856d5a4ea09a0cef9cf24d3aff)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:13.212088 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedde53d-d0c3-4654-88af-562fae334fa7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:13.213757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 155/314] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:13 2022] GET /identity/v3/users/d196ee856d5a4ea09a0cef9cf24d3aff => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:13.220600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedde53d-d0c3-4654-88af-562fae334fa7 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:13.257735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb48fc8-643b-41e1-b39c-3400c807b0fd None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:13.291927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ffd43-830e-4e33-b000-e070392a2e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:13.292814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffd43-830e-4e33-b000-e070392a2e21 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:13.293133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffd43-830e-4e33-b000-e070392a2e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:13.293461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ffd43-830e-4e33-b000-e070392a2e21 None admin] PATH_INFO: `/v3/system/users/d9cce33f079b4fc7ad3e8ceed52c44ac/roles/67832478d7b8428d996975409814f845` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:13.295802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ffd43-830e-4e33-b000-e070392a2e21 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d9cce33f079b4fc7ad3e8ceed52c44ac, role_id=67832478d7b8428d996975409814f845)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:13.297200 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ffd43-830e-4e33-b000-e070392a2e21 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:13.308164 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 160/315] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 9 15:02:13 2022] PUT /identity/v3/system/users/d9cce33f079b4fc7ad3e8ceed52c44ac/roles/67832478d7b8428d996975409814f845 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:13.333708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedde53d-d0c3-4654-88af-562fae334fa7 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:13.379084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504ceec-17c7-49c1-b883-e9e218f452d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:13.380420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504ceec-17c7-49c1-b883-e9e218f452d5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:13.380911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504ceec-17c7-49c1-b883-e9e218f452d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:13.381420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504ceec-17c7-49c1-b883-e9e218f452d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:13.456251 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504ceec-17c7-49c1-b883-e9e218f452d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:13.458255 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504ceec-17c7-49c1-b883-e9e218f452d5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:13.560208 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 156/316] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:13.566048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ffd43-830e-4e33-b000-e070392a2e21 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:13.664779 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ffd43-830e-4e33-b000-e070392a2e21 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:13.698475 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb5d02-e919-486b-9949-0921614df278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:13.699325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5d02-e919-486b-9949-0921614df278 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:13.699601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5d02-e919-486b-9949-0921614df278 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:13.699872 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5d02-e919-486b-9949-0921614df278 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:13.701129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5d02-e919-486b-9949-0921614df278 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:13.702569 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb5d02-e919-486b-9949-0921614df278 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:13.709095 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 161/317] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:13 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:13.710400 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504ceec-17c7-49c1-b883-e9e218f452d5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:13.812406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504ceec-17c7-49c1-b883-e9e218f452d5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:13.855332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e1706-d095-405a-be31-d730444449c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:13.857966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1706-d095-405a-be31-d730444449c0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:13.858272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1706-d095-405a-be31-d730444449c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:13.858547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1706-d095-405a-be31-d730444449c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:13.859429 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e1706-d095-405a-be31-d730444449c0 None admin] RBAC: Adding query filter params (role.id=7d37d7b5e03f435fbba6f4b1ee34cf55, scope.project.id=a2350e443c204e6d9d190688211b1729, user.id=d196ee856d5a4ea09a0cef9cf24d3aff) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:13.859743 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e1706-d095-405a-be31-d730444449c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:13.861253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e1706-d095-405a-be31-d730444449c0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:13.868271 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 157/318] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:13 2022] GET /identity/v3/role_assignments?user.id=d196ee856d5a4ea09a0cef9cf24d3aff&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:14.881868 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2f7d6-47bb-4976-bfe1-a6b84b084e3f None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:14.882383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2f7d6-47bb-4976-bfe1-a6b84b084e3f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:14.882487 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2f7d6-47bb-4976-bfe1-a6b84b084e3f None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:14.883915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 162/319] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:14.889471 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3ee3-4b43-42e3-b47a-6fec05e2d027 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:14.889850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3ee3-4b43-42e3-b47a-6fec05e2d027 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:14.890007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3ee3-4b43-42e3-b47a-6fec05e2d027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:14.906170 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-7cbb3ee3-4b43-42e3-b47a-6fec05e2d027 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:14.907349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-7cbb3ee3-4b43-42e3-b47a-6fec05e2d027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:14.969837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 158/320] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:14.976209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a4b2e-635e-4123-b63b-82624cf4fecf None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:14.976502 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a4b2e-635e-4123-b63b-82624cf4fecf None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:14.976783 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a4b2e-635e-4123-b63b-82624cf4fecf None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:14.978164 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 163/321] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:14.983825 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd8749-f924-45c5-a0cf-645f448f5a2c None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:14.984112 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd8749-f924-45c5-a0cf-645f448f5a2c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:14.984386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd8749-f924-45c5-a0cf-645f448f5a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:14.992473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a4b2e-635e-4123-b63b-82624cf4fecf None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:15.001235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-f9bd8749-f924-45c5-a0cf-645f448f5a2c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:15.002412 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-f9bd8749-f924-45c5-a0cf-645f448f5a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:15.066916 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 159/322] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:15.089272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd8749-f924-45c5-a0cf-645f448f5a2c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:15.095109 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a4b2e-635e-4123-b63b-82624cf4fecf None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:15.134443 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b37196-4db5-4878-8ac9-fb492374278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:15.135171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b37196-4db5-4878-8ac9-fb492374278c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.135449 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b37196-4db5-4878-8ac9-fb492374278c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.135719 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b37196-4db5-4878-8ac9-fb492374278c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.136864 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b37196-4db5-4878-8ac9-fb492374278c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:15.138983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b37196-4db5-4878-8ac9-fb492374278c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:15.140600 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 164/323] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:15.146143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b37196-4db5-4878-8ac9-fb492374278c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:15.191507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd8749-f924-45c5-a0cf-645f448f5a2c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:15.224792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7dae8-d96d-4112-a8dd-13dbb9ea77f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:15.225632 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7dae8-d96d-4112-a8dd-13dbb9ea77f9 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.225908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7dae8-d96d-4112-a8dd-13dbb9ea77f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.226182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7dae8-d96d-4112-a8dd-13dbb9ea77f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.236229 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7dae8-d96d-4112-a8dd-13dbb9ea77f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:15.243520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b37196-4db5-4878-8ac9-fb492374278c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-96f7dae8-d96d-4112-a8dd-13dbb9ea77f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:15.244002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:15.244890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:15.245978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 160/324] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:15.250766 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7dae8-d96d-4112-a8dd-13dbb9ea77f9 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:15.279625 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14e65-5f48-4351-8346-11a5d3737103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:15.280274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14e65-5f48-4351-8346-11a5d3737103 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.280560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14e65-5f48-4351-8346-11a5d3737103 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.280836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14e65-5f48-4351-8346-11a5d3737103 None admin] PATH_INFO: `/v3/groups` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.281560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14e65-5f48-4351-8346-11a5d3737103 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:15.283002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14e65-5f48-4351-8346-11a5d3737103 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:15.297804 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 165/325] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 9 15:02:15 2022] POST /identity/v3/groups => generated 226 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:02:15.355273 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7dae8-d96d-4112-a8dd-13dbb9ea77f9 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:15.391911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:15.392510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.392792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.393090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.393870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:15.394179 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:15.395659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:15.406103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 161/326] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:15.411953 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14e65-5f48-4351-8346-11a5d3737103 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:15.530367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14e65-5f48-4351-8346-11a5d3737103 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:15.567354 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e54298-486b-4c77-9f15-8fe459745c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:15.568239 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e54298-486b-4c77-9f15-8fe459745c2c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.568517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e54298-486b-4c77-9f15-8fe459745c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.568790 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e54298-486b-4c77-9f15-8fe459745c2c None admin] PATH_INFO: `/v3/users/46f9f0cc4880475884563fdce7899679` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.570014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e54298-486b-4c77-9f15-8fe459745c2c None admin] RBAC: Authorizing `identity:get_user(user_id=46f9f0cc4880475884563fdce7899679)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:15.571471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e54298-486b-4c77-9f15-8fe459745c2c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:15.573161 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 166/327] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:15 2022] GET /identity/v3/users/46f9f0cc4880475884563fdce7899679 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:15.579306 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:15.601595 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5fcec-f14d-46f6-b55f-069cf1e3e248 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.602153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5fcec-f14d-46f6-b55f-069cf1e3e248 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.602602 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5fcec-f14d-46f6-b55f-069cf1e3e248 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.604425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 167/328] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:15.609369 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceaad-b158-4c85-8bc7-5671f82c7d7c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.609913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceaad-b158-4c85-8bc7-5671f82c7d7c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.610196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cceaad-b158-4c85-8bc7-5671f82c7d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.628867 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-10cceaad-b158-4c85-8bc7-5671f82c7d7c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:15.630585 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-10cceaad-b158-4c85-8bc7-5671f82c7d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:15.697756 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49584f6-3681-4a44-aed0-e8518569f3b3 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:15.700714 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 168/329] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:15.726403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cceaad-b158-4c85-8bc7-5671f82c7d7c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:15.732849 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a1619-5e97-44c0-a18c-3c988baa4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:15.733754 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a1619-5e97-44c0-a18c-3c988baa4cf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.734039 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a1619-5e97-44c0-a18c-3c988baa4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.734319 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a1619-5e97-44c0-a18c-3c988baa4cf8 None admin] PATH_INFO: `/v3/system/users/46f9f0cc4880475884563fdce7899679/roles/7d37d7b5e03f435fbba6f4b1ee34cf55` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.736264 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a1619-5e97-44c0-a18c-3c988baa4cf8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=46f9f0cc4880475884563fdce7899679, role_id=7d37d7b5e03f435fbba6f4b1ee34cf55)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:15.738076 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a1619-5e97-44c0-a18c-3c988baa4cf8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:15.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 162/330] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 9 15:02:15 2022] PUT /identity/v3/system/users/46f9f0cc4880475884563fdce7899679/roles/7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:15.846197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cceaad-b158-4c85-8bc7-5671f82c7d7c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:15.884975 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3823b7a-6087-487d-8766-9fc62c482370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:15.886610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3823b7a-6087-487d-8766-9fc62c482370 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:15.887155 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3823b7a-6087-487d-8766-9fc62c482370 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:15.887683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3823b7a-6087-487d-8766-9fc62c482370 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:15.901702 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3823b7a-6087-487d-8766-9fc62c482370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-d3823b7a-6087-487d-8766-9fc62c482370 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:15.910097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:15.911071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:15.911855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:15.914721 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 169/331] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:15.921496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a1619-5e97-44c0-a18c-3c988baa4cf8 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:16.027555 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a1619-5e97-44c0-a18c-3c988baa4cf8 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:16.064231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:16.064912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:16.065232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:16.065507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:16.066267 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:16.066571 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:16.067941 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:16.078293 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 163/332] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:16.083728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3823b7a-6087-487d-8766-9fc62c482370 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:16.216565 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3823b7a-6087-487d-8766-9fc62c482370 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:16.253775 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a359-2cd6-4cbb-b278-3cfd65376e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:16.254657 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a359-2cd6-4cbb-b278-3cfd65376e14 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:16.255192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a359-2cd6-4cbb-b278-3cfd65376e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:16.255468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a359-2cd6-4cbb-b278-3cfd65376e14 None admin] PATH_INFO: `/v3/users/d9cce33f079b4fc7ad3e8ceed52c44ac` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:16.256683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894a359-2cd6-4cbb-b278-3cfd65376e14 None admin] RBAC: Authorizing `identity:get_user(user_id=d9cce33f079b4fc7ad3e8ceed52c44ac)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:16.258176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894a359-2cd6-4cbb-b278-3cfd65376e14 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:16.260198 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 170/333] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:16 2022] GET /identity/v3/users/d9cce33f079b4fc7ad3e8ceed52c44ac => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:16.266353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:16.386768 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcef881-44d5-467b-a8cf-9b068d476f26 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:16.423254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18298bd1-3b58-4366-9e50-a043db7b363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:16.424153 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298bd1-3b58-4366-9e50-a043db7b363b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:16.424579 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298bd1-3b58-4366-9e50-a043db7b363b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:16.425025 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298bd1-3b58-4366-9e50-a043db7b363b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:16.426075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18298bd1-3b58-4366-9e50-a043db7b363b None admin] RBAC: Adding query filter params (role.id=67832478d7b8428d996975409814f845, user.id=d9cce33f079b4fc7ad3e8ceed52c44ac, scope.system=all) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:16.426528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18298bd1-3b58-4366-9e50-a043db7b363b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:16.428092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18298bd1-3b58-4366-9e50-a043db7b363b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:16.440452 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 164/334] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:02:16 2022] GET /identity/v3/role_assignments?user.id=d9cce33f079b4fc7ad3e8ceed52c44ac&scope.system=all&role.id=67832478d7b8428d996975409814f845 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:17.988461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf47d02-1216-474a-9213-f216ec67acf4 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:17.989022 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf47d02-1216-474a-9213-f216ec67acf4 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:17.989022 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf47d02-1216-474a-9213-f216ec67acf4 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:17.990643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 171/335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:17.996654 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d2b41-0181-493f-8af5-3f84907a6891 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:17.998089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d2b41-0181-493f-8af5-3f84907a6891 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:17.998567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d2b41-0181-493f-8af5-3f84907a6891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:18.016875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-758d2b41-0181-493f-8af5-3f84907a6891 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:18.018415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-758d2b41-0181-493f-8af5-3f84907a6891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:18.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 165/336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:18.117549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf47d02-1216-474a-9213-f216ec67acf4 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:18.239061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf47d02-1216-474a-9213-f216ec67acf4 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:18.275834 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a53371-f0a7-4cba-87be-1308080e65e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:18.276981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a53371-f0a7-4cba-87be-1308080e65e6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:18.277520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a53371-f0a7-4cba-87be-1308080e65e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:18.278557 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a53371-f0a7-4cba-87be-1308080e65e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:18.293975 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a53371-f0a7-4cba-87be-1308080e65e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-12a53371-f0a7-4cba-87be-1308080e65e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:18.302125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:18.303037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:18.303863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:18.306274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 172/337] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:18.312692 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d2b41-0181-493f-8af5-3f84907a6891 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:18.429965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d2b41-0181-493f-8af5-3f84907a6891 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:18.469833 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:18.470865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:18.471362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:18.471888 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:18.472844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:18.473374 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:18.475011 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:18.484722 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 166/338] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:18.489993 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a53371-f0a7-4cba-87be-1308080e65e6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:18.647154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a53371-f0a7-4cba-87be-1308080e65e6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:18.693081 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e26658-19b1-4874-86d4-dd7fc383a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:18.693735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26658-19b1-4874-86d4-dd7fc383a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:18.694009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26658-19b1-4874-86d4-dd7fc383a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:18.694276 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26658-19b1-4874-86d4-dd7fc383a2e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:18.773858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e26658-19b1-4874-86d4-dd7fc383a2e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:18.775268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e26658-19b1-4874-86d4-dd7fc383a2e6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:18.908940 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 173/339] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:18.916408 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:19.046741 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897a69-eee5-489c-b6f0-b5d15dbcaf5b None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.047242 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897a69-eee5-489c-b6f0-b5d15dbcaf5b None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.047242 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7897a69-eee5-489c-b6f0-b5d15dbcaf5b None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.048606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 174/340] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:19.053886 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47d78c-2bff-4fb7-b4fd-2aed2c4a7329 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.054178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47d78c-2bff-4fb7-b4fd-2aed2c4a7329 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.054639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47d78c-2bff-4fb7-b4fd-2aed2c4a7329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.060957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd055a-9765-4c48-9207-d7f5738d9c6c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:19.090094 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-8f47d78c-2bff-4fb7-b4fd-2aed2c4a7329 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:19.091343 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-8f47d78c-2bff-4fb7-b4fd-2aed2c4a7329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:19.100615 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8148f-c48b-451c-8242-1fc0f0cd8dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:19.101829 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8148f-c48b-451c-8242-1fc0f0cd8dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.102323 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8148f-c48b-451c-8242-1fc0f0cd8dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.102800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8148f-c48b-451c-8242-1fc0f0cd8dc2 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.106234 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8148f-c48b-451c-8242-1fc0f0cd8dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:19.109097 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8148f-c48b-451c-8242-1fc0f0cd8dc2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:19.113663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 167/341] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:18 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:19.125472 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8148f-c48b-451c-8242-1fc0f0cd8dc2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:19.220808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 175/342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:19.223292 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b482f0-66db-4f70-8138-e217e5ddac00 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.223987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b482f0-66db-4f70-8138-e217e5ddac00 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.224660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b482f0-66db-4f70-8138-e217e5ddac00 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.227611 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 176/343] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:19.233031 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebb1b4-596f-4f22-a5b2-1389bdafb5c1 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.233622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebb1b4-596f-4f22-a5b2-1389bdafb5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.234111 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ebb1b4-596f-4f22-a5b2-1389bdafb5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.260968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8148f-c48b-451c-8242-1fc0f0cd8dc2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:19.270135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-59ebb1b4-596f-4f22-a5b2-1389bdafb5c1 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:19.271585 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-59ebb1b4-596f-4f22-a5b2-1389bdafb5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:19.300398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542b61b-7781-41ad-b1f7-30678a11504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:19.301902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542b61b-7781-41ad-b1f7-30678a11504e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.302348 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542b61b-7781-41ad-b1f7-30678a11504e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.302913 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542b61b-7781-41ad-b1f7-30678a11504e None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.310115 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542b61b-7781-41ad-b1f7-30678a11504e None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:19.312090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542b61b-7781-41ad-b1f7-30678a11504e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:19.314736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 168/344] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:19 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:19.315867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542b61b-7781-41ad-b1f7-30678a11504e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:19.411793 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 177/345] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:19.415737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebb1b4-596f-4f22-a5b2-1389bdafb5c1 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:19.480377 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542b61b-7781-41ad-b1f7-30678a11504e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:19.520090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba0ed4-1da6-4857-94f5-f98076b50ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:19.520887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba0ed4-1da6-4857-94f5-f98076b50ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.521192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba0ed4-1da6-4857-94f5-f98076b50ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.521460 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba0ed4-1da6-4857-94f5-f98076b50ba8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.543167 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba0ed4-1da6-4857-94f5-f98076b50ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-31ba0ed4-1da6-4857-94f5-f98076b50ba8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:19.551130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:19.552007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:19.552810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 169/346] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:19.560341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba0ed4-1da6-4857-94f5-f98076b50ba8 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:19.607187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ebb1b4-596f-4f22-a5b2-1389bdafb5c1 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:19.656801 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:19.657540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.657826 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.658119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.658941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] RBAC: Adding query filter params (group.id=fcd9246af06d4109a55ee05e6fc4437b, role.id=67832478d7b8428d996975409814f845, scope.project.id=8f180df64cf54d3996588ba6518e4e4c) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:19.659248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:19.661725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:19.670422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 178/347] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:19 2022] GET /identity/v3/role_assignments?group.id=fcd9246af06d4109a55ee05e6fc4437b&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=67832478d7b8428d996975409814f845 => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:19.677364 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:19.683387 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba0ed4-1da6-4857-94f5-f98076b50ba8 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:19.757860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b927556-a2b7-48c7-ad0e-e513e085d164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:19.759059 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b927556-a2b7-48c7-ad0e-e513e085d164 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.759541 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b927556-a2b7-48c7-ad0e-e513e085d164 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.759985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b927556-a2b7-48c7-ad0e-e513e085d164 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.786124 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b927556-a2b7-48c7-ad0e-e513e085d164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-7b927556-a2b7-48c7-ad0e-e513e085d164 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:19.795217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:19.796196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:19.797057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:19.799544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca0ef0-056f-4468-882c-186550267108 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:19.800171 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 170/348] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:19.801836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016c1cd-ce38-47cc-97db-ace4cde45663 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.802328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016c1cd-ce38-47cc-97db-ace4cde45663 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.802812 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016c1cd-ce38-47cc-97db-ace4cde45663 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.804771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 171/349] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:19.805793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016c1cd-ce38-47cc-97db-ace4cde45663 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:19.867452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678d183-af52-47b5-b558-2958f48cc72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:19.869492 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678d183-af52-47b5-b558-2958f48cc72d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.870047 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678d183-af52-47b5-b558-2958f48cc72d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.870664 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678d183-af52-47b5-b558-2958f48cc72d None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.871718 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678d183-af52-47b5-b558-2958f48cc72d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:19.875883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678d183-af52-47b5-b558-2958f48cc72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:19.877744 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678d183-af52-47b5-b558-2958f48cc72d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:19.900000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 179/350] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:19.901892 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4f28d-6d0a-4494-a45f-bb00c65b00b5 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:19.902394 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4f28d-6d0a-4494-a45f-bb00c65b00b5 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:19.903417 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4f28d-6d0a-4494-a45f-bb00c65b00b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:19.921973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-39e4f28d-6d0a-4494-a45f-bb00c65b00b5 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:19.923763 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-39e4f28d-6d0a-4494-a45f-bb00c65b00b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:19.994333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016c1cd-ce38-47cc-97db-ace4cde45663 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:20.020964 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 180/351] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:20.026009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4f28d-6d0a-4494-a45f-bb00c65b00b5 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:20.046652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:20.047708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.048286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.048785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.051186 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:20.051812 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:20.054712 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:20.068112 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 172/352] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:20.074807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:20.167359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4f28d-6d0a-4494-a45f-bb00c65b00b5 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:20.208698 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0391054-457c-4832-9667-d66f0660ee1b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:20.214229 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037a8c0-8513-4cef-9187-ef8b04215ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:20.215106 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037a8c0-8513-4cef-9187-ef8b04215ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.215420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037a8c0-8513-4cef-9187-ef8b04215ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.215693 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037a8c0-8513-4cef-9187-ef8b04215ac7 None admin] PATH_INFO: `/v3/users/46f9f0cc4880475884563fdce7899679` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.217793 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037a8c0-8513-4cef-9187-ef8b04215ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=46f9f0cc4880475884563fdce7899679)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:20.219429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037a8c0-8513-4cef-9187-ef8b04215ac7 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:20.225169 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 181/353] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:02:20 2022] GET /identity/v3/users/46f9f0cc4880475884563fdce7899679 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:20.226296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037a8c0-8513-4cef-9187-ef8b04215ac7 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:20.251896 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d179d40-2ced-436b-8da0-cbebd7234bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:20.252695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d179d40-2ced-436b-8da0-cbebd7234bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.252974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d179d40-2ced-436b-8da0-cbebd7234bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.253277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d179d40-2ced-436b-8da0-cbebd7234bb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.268897 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d179d40-2ced-436b-8da0-cbebd7234bb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-0d179d40-2ced-436b-8da0-cbebd7234bb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:20.278971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:20.279872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 173/354] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:20.280666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708372fe-2653-4f70-93e9-cc307392a92c None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.281426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708372fe-2653-4f70-93e9-cc307392a92c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.281426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708372fe-2653-4f70-93e9-cc307392a92c None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.282931 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 174/355] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:20.283885 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708372fe-2653-4f70-93e9-cc307392a92c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:20.346172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037a8c0-8513-4cef-9187-ef8b04215ac7 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:20.383109 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8b8c2-4c98-4466-94c3-545c80f28b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:20.384134 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8b8c2-4c98-4466-94c3-545c80f28b92 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.384622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8b8c2-4c98-4466-94c3-545c80f28b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.385126 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8b8c2-4c98-4466-94c3-545c80f28b92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.407355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708372fe-2653-4f70-93e9-cc307392a92c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:20.450218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26dd90-e99a-43f9-aed0-41f3275fd886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:20.451256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26dd90-e99a-43f9-aed0-41f3275fd886 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.451744 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26dd90-e99a-43f9-aed0-41f3275fd886 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.452213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26dd90-e99a-43f9-aed0-41f3275fd886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.453274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26dd90-e99a-43f9-aed0-41f3275fd886 None admin] RBAC: Adding query filter params (role.id=7d37d7b5e03f435fbba6f4b1ee34cf55, user.id=46f9f0cc4880475884563fdce7899679, scope.system=all) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:20.453782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26dd90-e99a-43f9-aed0-41f3275fd886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:20.455410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26dd90-e99a-43f9-aed0-41f3275fd886 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:20.472569 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 175/356] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:02:20 2022] GET /identity/v3/role_assignments?user.id=46f9f0cc4880475884563fdce7899679&scope.system=all&role.id=7d37d7b5e03f435fbba6f4b1ee34cf55 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:20.472894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8b8c2-4c98-4466-94c3-545c80f28b92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:20.474564 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78466-7d91-498b-acd9-23388d31ddbd None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.474858 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78466-7d91-498b-acd9-23388d31ddbd None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.475135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78466-7d91-498b-acd9-23388d31ddbd None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.475996 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8b8c2-4c98-4466-94c3-545c80f28b92 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:20.476681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 176/357] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:20.481085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa78466-7d91-498b-acd9-23388d31ddbd None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:20.605094 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 182/358] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:20.606498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552bc3a-ae6e-49ae-b6a0-34b742aeb51a None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.606876 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552bc3a-ae6e-49ae-b6a0-34b742aeb51a None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.607248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552bc3a-ae6e-49ae-b6a0-34b742aeb51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.620962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa78466-7d91-498b-acd9-23388d31ddbd None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:20.626723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-1552bc3a-ae6e-49ae-b6a0-34b742aeb51a None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:20.628072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-1552bc3a-ae6e-49ae-b6a0-34b742aeb51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:20.675344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd006928-7051-4da2-86ed-fa3c933790a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:20.676025 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006928-7051-4da2-86ed-fa3c933790a8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.676298 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006928-7051-4da2-86ed-fa3c933790a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.676565 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd006928-7051-4da2-86ed-fa3c933790a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.677404 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd006928-7051-4da2-86ed-fa3c933790a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:20.677715 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd006928-7051-4da2-86ed-fa3c933790a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:20.679091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd006928-7051-4da2-86ed-fa3c933790a8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:20.692526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 177/359] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:20.693861 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854ac7-0569-4906-ae3e-777b25e016e3 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.694149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854ac7-0569-4906-ae3e-777b25e016e3 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.694429 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f854ac7-0569-4906-ae3e-777b25e016e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.714154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-3f854ac7-0569-4906-ae3e-777b25e016e3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:20.715891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-3f854ac7-0569-4906-ae3e-777b25e016e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:20.722852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 183/360] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:20.723969 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552bc3a-ae6e-49ae-b6a0-34b742aeb51a None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:20.811487 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 178/361] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:20.812442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f854ac7-0569-4906-ae3e-777b25e016e3 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:20.858570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552bc3a-ae6e-49ae-b6a0-34b742aeb51a None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:20.898045 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a949-cf25-40df-935d-ce952a09f73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:20.898902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a949-cf25-40df-935d-ce952a09f73d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.899175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a949-cf25-40df-935d-ce952a09f73d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.899444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a949-cf25-40df-935d-ce952a09f73d None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:20.900621 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a949-cf25-40df-935d-ce952a09f73d None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:20.902091 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a949-cf25-40df-935d-ce952a09f73d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:20.904362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 184/362] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:20 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:20.905093 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a949-cf25-40df-935d-ce952a09f73d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:20.930970 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f854ac7-0569-4906-ae3e-777b25e016e3 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:20.976892 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f9cef-22d4-41ec-9d58-4740accaa4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:20.978304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f9cef-22d4-41ec-9d58-4740accaa4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:20.978304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f9cef-22d4-41ec-9d58-4740accaa4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:20.978304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f9cef-22d4-41ec-9d58-4740accaa4d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.033816 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a949-cf25-40df-935d-ce952a09f73d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.059173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f9cef-22d4-41ec-9d58-4740accaa4d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:21.061183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f9cef-22d4-41ec-9d58-4740accaa4d4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:21.089636 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2eef5-6dd7-40f0-96e3-73c0f92e26ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.090484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2eef5-6dd7-40f0-96e3-73c0f92e26ba None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.090765 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2eef5-6dd7-40f0-96e3-73c0f92e26ba None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.091044 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2eef5-6dd7-40f0-96e3-73c0f92e26ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.102393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2eef5-6dd7-40f0-96e3-73c0f92e26ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-49b2eef5-6dd7-40f0-96e3-73c0f92e26ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:21.113455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:21.114359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:21.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 185/363] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:21.115827 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2eef5-6dd7-40f0-96e3-73c0f92e26ba None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.186955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 179/364] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:21.188428 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f9cef-22d4-41ec-9d58-4740accaa4d4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.240107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2eef5-6dd7-40f0-96e3-73c0f92e26ba None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.295939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53262e60-252f-4a1c-9c01-2ca885153e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.295939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53262e60-252f-4a1c-9c01-2ca885153e7f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.295939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53262e60-252f-4a1c-9c01-2ca885153e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.296479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53262e60-252f-4a1c-9c01-2ca885153e7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.313750 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53262e60-252f-4a1c-9c01-2ca885153e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:21.320948 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f9cef-22d4-41ec-9d58-4740accaa4d4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-53262e60-252f-4a1c-9c01-2ca885153e7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:21.321420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:21.322444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:21.323252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 186/365] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:21.325368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53262e60-252f-4a1c-9c01-2ca885153e7f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.374421 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f9f8b-a0fa-42ba-b200-80312435715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.375222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f9f8b-a0fa-42ba-b200-80312435715d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.375496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f9f8b-a0fa-42ba-b200-80312435715d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.375762 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f9f8b-a0fa-42ba-b200-80312435715d None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.377089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f9f8b-a0fa-42ba-b200-80312435715d None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:21.378791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f9f8b-a0fa-42ba-b200-80312435715d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:21.380963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 180/366] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:21.382145 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f9f8b-a0fa-42ba-b200-80312435715d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.442674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53262e60-252f-4a1c-9c01-2ca885153e7f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.489834 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f9f8b-a0fa-42ba-b200-80312435715d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.490167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.490771 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.491318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.492067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:21.492370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:21.493767 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:21.502586 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 187/367] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:21.503391 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.538485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8fb50-9519-4521-a39d-558eb47caa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.539345 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8fb50-9519-4521-a39d-558eb47caa0b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.539620 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8fb50-9519-4521-a39d-558eb47caa0b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.539887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8fb50-9519-4521-a39d-558eb47caa0b None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.541079 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8fb50-9519-4521-a39d-558eb47caa0b None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:21.542508 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8fb50-9519-4521-a39d-558eb47caa0b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:21.545024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 181/368] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:21.546818 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8fb50-9519-4521-a39d-558eb47caa0b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.627712 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cd94b-0c11-404b-b5bd-2c70f0578a46 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.670538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.670538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.670538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.670538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.671174 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:21.671174 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:21.671515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:21.682099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 188/369] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:21.682942 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.691149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8fb50-9519-4521-a39d-558eb47caa0b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.736314 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.737154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.737438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.737713 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.738605 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] RBAC: Adding query filter params (group.id=fcd9246af06d4109a55ee05e6fc4437b, role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=8f180df64cf54d3996588ba6518e4e4c) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:21.738924 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:21.740369 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:21.746542 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 182/370] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/role_assignments?group.id=fcd9246af06d4109a55ee05e6fc4437b&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:21.747908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.817073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb3d49-dd93-4ac1-b0cc-a750d56c82d5 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.863451 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5fbb9-bda4-4fe6-a7e6-80ae3d946c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.864436 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5fbb9-bda4-4fe6-a7e6-80ae3d946c41 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.864836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5fbb9-bda4-4fe6-a7e6-80ae3d946c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.865168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5fbb9-bda4-4fe6-a7e6-80ae3d946c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.879242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba71d-3ef7-442d-8b9a-023c64f4e430 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:21.924513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfda7a-385b-4902-b785-6df06ce1ba66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:21.925435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfda7a-385b-4902-b785-6df06ce1ba66 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:21.925772 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfda7a-385b-4902-b785-6df06ce1ba66 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:21.926049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfda7a-385b-4902-b785-6df06ce1ba66 None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:21.927355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfda7a-385b-4902-b785-6df06ce1ba66 None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:21.928878 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfda7a-385b-4902-b785-6df06ce1ba66 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:21.930930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 183/371] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:21.934940 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfda7a-385b-4902-b785-6df06ce1ba66 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:21.956672 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5fbb9-bda4-4fe6-a7e6-80ae3d946c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:21.958971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5fbb9-bda4-4fe6-a7e6-80ae3d946c41 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.047749 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfda7a-385b-4902-b785-6df06ce1ba66 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.081915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78175edf-bafc-49c2-8c94-0f4d88bdca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.082290 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 189/372] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:22.082558 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78175edf-bafc-49c2-8c94-0f4d88bdca89 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.082840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78175edf-bafc-49c2-8c94-0f4d88bdca89 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.083109 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78175edf-bafc-49c2-8c94-0f4d88bdca89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.083772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5fbb9-bda4-4fe6-a7e6-80ae3d946c41 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.157736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78175edf-bafc-49c2-8c94-0f4d88bdca89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:22.159206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78175edf-bafc-49c2-8c94-0f4d88bdca89 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.190096 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5fbb9-bda4-4fe6-a7e6-80ae3d946c41 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.224431 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.225110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.225393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.225665 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.226502 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] RBAC: Adding query filter params (group.id=fcd9246af06d4109a55ee05e6fc4437b, role.id=67832478d7b8428d996975409814f845, scope.project.id=a2350e443c204e6d9d190688211b1729) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:22.226813 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:22.228264 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.234305 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 190/373] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/role_assignments?group.id=fcd9246af06d4109a55ee05e6fc4437b&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=67832478d7b8428d996975409814f845 => generated 260 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:22.234936 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.265473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 184/374] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:22.268630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78175edf-bafc-49c2-8c94-0f4d88bdca89 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.356481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b9777-0d8d-41ef-b939-069ec24b9388 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.394321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65997c4-0ea8-4e42-8356-186f210ad5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65997c4-0ea8-4e42-8356-186f210ad5fd None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.395390 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65997c4-0ea8-4e42-8356-186f210ad5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.395664 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65997c4-0ea8-4e42-8356-186f210ad5fd None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.397081 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65997c4-0ea8-4e42-8356-186f210ad5fd None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:22.398094 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78175edf-bafc-49c2-8c94-0f4d88bdca89 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.398549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65997c4-0ea8-4e42-8356-186f210ad5fd None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.400505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 191/375] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:22.401658 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0089-a44d-4c16-ae13-42d0131082c5 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.401939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0089-a44d-4c16-ae13-42d0131082c5 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.402212 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0089-a44d-4c16-ae13-42d0131082c5 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.403863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 192/376] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:22.407673 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d0089-a44d-4c16-ae13-42d0131082c5 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.442835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e05c5-04b3-471d-9667-86c838c6bd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.443707 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e05c5-04b3-471d-9667-86c838c6bd56 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.443985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e05c5-04b3-471d-9667-86c838c6bd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.449084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e05c5-04b3-471d-9667-86c838c6bd56 None admin] PATH_INFO: `/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.449084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e05c5-04b3-471d-9667-86c838c6bd56 None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51f62b70d42e281dc8a352a6f16d6)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:22.449084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e05c5-04b3-471d-9667-86c838c6bd56 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.450891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 185/377] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6 => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:22.452135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1ef8f-047e-449e-8484-9b6a13ab10e3 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.452413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1ef8f-047e-449e-8484-9b6a13ab10e3 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.452683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1ef8f-047e-449e-8484-9b6a13ab10e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.471041 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-30e1ef8f-047e-449e-8484-9b6a13ab10e3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:22.472268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-30e1ef8f-047e-449e-8484-9b6a13ab10e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:22.565203 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 186/378] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:22.566419 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1ef8f-047e-449e-8484-9b6a13ab10e3 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.572781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d0089-a44d-4c16-ae13-42d0131082c5 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.632677 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8198e6e-4fcf-4ae7-b02a-e14d9f832eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.633953 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198e6e-4fcf-4ae7-b02a-e14d9f832eba None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.634506 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198e6e-4fcf-4ae7-b02a-e14d9f832eba None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.634787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198e6e-4fcf-4ae7-b02a-e14d9f832eba None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.636159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8198e6e-4fcf-4ae7-b02a-e14d9f832eba None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:22.638516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8198e6e-4fcf-4ae7-b02a-e14d9f832eba None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.640763 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 193/379] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:22.641867 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8198e6e-4fcf-4ae7-b02a-e14d9f832eba None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.691369 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1ef8f-047e-449e-8484-9b6a13ab10e3 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.734619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53c64-75f0-4968-9b58-c5d27f14e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.735477 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53c64-75f0-4968-9b58-c5d27f14e41a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.735752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53c64-75f0-4968-9b58-c5d27f14e41a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.736019 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53c64-75f0-4968-9b58-c5d27f14e41a None admin] PATH_INFO: `/v3/groups/a92cc2c5c08b4c3a97754cf4fafabf60` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.741335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53c64-75f0-4968-9b58-c5d27f14e41a None admin] RBAC: Authorizing `identity:get_group(group_id=a92cc2c5c08b4c3a97754cf4fafabf60)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:22.742762 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e53c64-75f0-4968-9b58-c5d27f14e41a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.745578 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 187/380] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/groups/a92cc2c5c08b4c3a97754cf4fafabf60 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:22.746393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53c64-75f0-4968-9b58-c5d27f14e41a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.767005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8198e6e-4fcf-4ae7-b02a-e14d9f832eba None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.805800 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d86ee-9f3f-4ebe-88c7-e6aa16374215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.806665 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d86ee-9f3f-4ebe-88c7-e6aa16374215 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.806947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d86ee-9f3f-4ebe-88c7-e6aa16374215 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.807226 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d86ee-9f3f-4ebe-88c7-e6aa16374215 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.817638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659d86ee-9f3f-4ebe-88c7-e6aa16374215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-659d86ee-9f3f-4ebe-88c7-e6aa16374215 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:22.825377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:22.826281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:22.827104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 194/381] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:22.827760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d86ee-9f3f-4ebe-88c7-e6aa16374215 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.862445 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53c64-75f0-4968-9b58-c5d27f14e41a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.896952 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.897830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.898106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.898377 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.899215 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] RBAC: Adding query filter params (group.id=fcd9246af06d4109a55ee05e6fc4437b, role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=a2350e443c204e6d9d190688211b1729) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:22.899528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:22.901059 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.907241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 188/382] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/role_assignments?group.id=fcd9246af06d4109a55ee05e6fc4437b&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:22.908068 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:22.930277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d86ee-9f3f-4ebe-88c7-e6aa16374215 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:22.965738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:22.966334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:22.966617 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:22.966891 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:22.967742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] RBAC: Adding query filter params (group.id=a92cc2c5c08b4c3a97754cf4fafabf60, role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=f3f51f62b70d42e281dc8a352a6f16d6) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:22.968055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:22.969552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:22.975474 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 195/383] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/role_assignments?group.id=a92cc2c5c08b4c3a97754cf4fafabf60&scope.project.id=f3f51f62b70d42e281dc8a352a6f16d6&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 260 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:23.019670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd016c-5043-4950-bb80-c0c189ea5706 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:23.059595 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:23.060240 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:23.060510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:23.060776 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:23.061648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:23.061953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:23.063412 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:23.073303 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 189/384] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:23.079915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:23.196625 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480210c-19f9-4316-ad95-1a7c5e62c806 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:23.237449 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96083c7d-2dea-4acb-9d52-69497620a9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:23.238225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96083c7d-2dea-4acb-9d52-69497620a9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:23.238497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96083c7d-2dea-4acb-9d52-69497620a9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:23.238763 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96083c7d-2dea-4acb-9d52-69497620a9e1 None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:23.240626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96083c7d-2dea-4acb-9d52-69497620a9e1 None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:23.242179 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96083c7d-2dea-4acb-9d52-69497620a9e1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:23.244180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 196/385] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:23 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:23.250550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:23.391464 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da78e20-d147-4879-90ce-627afaf8e526 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:23.429090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787c870-6a1a-40b7-ba50-35fda910c350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:23.429960 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787c870-6a1a-40b7-ba50-35fda910c350 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:23.430240 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787c870-6a1a-40b7-ba50-35fda910c350 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:23.430516 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787c870-6a1a-40b7-ba50-35fda910c350 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:23.431698 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787c870-6a1a-40b7-ba50-35fda910c350 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:23.433167 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787c870-6a1a-40b7-ba50-35fda910c350 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:23.435218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 190/386] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:23 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:23.440660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96083c7d-2dea-4acb-9d52-69497620a9e1 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:23.578073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96083c7d-2dea-4acb-9d52-69497620a9e1 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:23.613913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f45b9-daca-4c80-9a5d-e6ca321153aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:23.614797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f45b9-daca-4c80-9a5d-e6ca321153aa None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:23.615075 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f45b9-daca-4c80-9a5d-e6ca321153aa None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:23.615349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f45b9-daca-4c80-9a5d-e6ca321153aa None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c/groups/fcd9246af06d4109a55ee05e6fc4437b/roles/67832478d7b8428d996975409814f845` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:23.617322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f45b9-daca-4c80-9a5d-e6ca321153aa None admin] RBAC: Authorizing `identity:create_grant(project_id=8f180df64cf54d3996588ba6518e4e4c, group_id=fcd9246af06d4109a55ee05e6fc4437b, role_id=67832478d7b8428d996975409814f845)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:23.618758 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f45b9-daca-4c80-9a5d-e6ca321153aa None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:23.645623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 197/387] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:02:23 2022] PUT /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c/groups/fcd9246af06d4109a55ee05e6fc4437b/roles/67832478d7b8428d996975409814f845 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:24.761235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928a2e-bf08-4e79-ac27-be0c145c4a98 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:24.761755 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928a2e-bf08-4e79-ac27-be0c145c4a98 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:24.761830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928a2e-bf08-4e79-ac27-be0c145c4a98 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:24.764013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 191/388] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:24.769208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cf7fd-81e0-48c0-be88-335a2dbb345b None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:24.770075 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cf7fd-81e0-48c0-be88-335a2dbb345b None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:24.770579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cf7fd-81e0-48c0-be88-335a2dbb345b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:24.794032 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-110cf7fd-81e0-48c0-be88-335a2dbb345b None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:24.795503 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-110cf7fd-81e0-48c0-be88-335a2dbb345b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:24.881645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 198/389] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:24.920579 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef928a2e-bf08-4e79-ac27-be0c145c4a98 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:24.977966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3938d-a510-4ec3-9184-27b8315a6adc None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:24.978640 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3938d-a510-4ec3-9184-27b8315a6adc None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:24.979113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3938d-a510-4ec3-9184-27b8315a6adc None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:24.980674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 199/390] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:02:24.986478 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea3392-b70f-4010-8607-d596f1cb545d None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:24.986761 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea3392-b70f-4010-8607-d596f1cb545d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:24.987036 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea3392-b70f-4010-8607-d596f1cb545d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.019021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-0cea3392-b70f-4010-8607-d596f1cb545d None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:25.020111 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-0cea3392-b70f-4010-8607-d596f1cb545d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:25.061967 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef928a2e-bf08-4e79-ac27-be0c145c4a98 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:25.095204 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79eaaac-e608-40ae-8d4a-0f0bdb4de848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:25.095998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eaaac-e608-40ae-8d4a-0f0bdb4de848 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.096278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eaaac-e608-40ae-8d4a-0f0bdb4de848 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.096555 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79eaaac-e608-40ae-8d4a-0f0bdb4de848 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.107171 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79eaaac-e608-40ae-8d4a-0f0bdb4de848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-d79eaaac-e608-40ae-8d4a-0f0bdb4de848 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:25.115213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:25.116105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:25.117507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 192/391] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:25.122731 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79eaaac-e608-40ae-8d4a-0f0bdb4de848 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:25.187601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 200/392] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:25.223640 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea3392-b70f-4010-8607-d596f1cb545d None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:25.224046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79eaaac-e608-40ae-8d4a-0f0bdb4de848 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:25.257335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:25.257996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.258274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.258545 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.259288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:25.259588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:25.260955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:25.269773 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 193/393] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:25.275239 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:25.376743 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7986ab-f61d-4699-b446-aa5f221e56cd None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:25.410785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f78eb-f6ec-4057-b267-f0ee9da1bc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:25.411630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f78eb-f6ec-4057-b267-f0ee9da1bc1d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.411955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f78eb-f6ec-4057-b267-f0ee9da1bc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.412230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f78eb-f6ec-4057-b267-f0ee9da1bc1d None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.413433 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f78eb-f6ec-4057-b267-f0ee9da1bc1d None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:25.414929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f78eb-f6ec-4057-b267-f0ee9da1bc1d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:25.421173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 194/394] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:25 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:25.422798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f78eb-f6ec-4057-b267-f0ee9da1bc1d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:25.470911 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea3392-b70f-4010-8607-d596f1cb545d None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:25.522399 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f78eb-f6ec-4057-b267-f0ee9da1bc1d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:25.559118 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d78967-249d-4dd3-88c4-443b886cfd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:25.560001 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d78967-249d-4dd3-88c4-443b886cfd06 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.560278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d78967-249d-4dd3-88c4-443b886cfd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.560548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d78967-249d-4dd3-88c4-443b886cfd06 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.560797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dec37c-41e0-4782-b913-835bf24d4826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:25.561857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dec37c-41e0-4782-b913-835bf24d4826 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.561981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d78967-249d-4dd3-88c4-443b886cfd06 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:25.562460 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dec37c-41e0-4782-b913-835bf24d4826 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.562929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dec37c-41e0-4782-b913-835bf24d4826 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.563384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d78967-249d-4dd3-88c4-443b886cfd06 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:25.565201 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 195/395] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:25 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:25.570842 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d78967-249d-4dd3-88c4-443b886cfd06 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:25.593973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dec37c-41e0-4782-b913-835bf24d4826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-57dec37c-41e0-4782-b913-835bf24d4826 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:25.609074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:25.609988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:25.610799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:25.613518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 201/396] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:25.618991 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dec37c-41e0-4782-b913-835bf24d4826 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:25.670836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d78967-249d-4dd3-88c4-443b886cfd06 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:25.704138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e21a6-fc83-47ac-9798-13f1732199a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:25.705018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e21a6-fc83-47ac-9798-13f1732199a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.705320 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e21a6-fc83-47ac-9798-13f1732199a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.705586 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e21a6-fc83-47ac-9798-13f1732199a3 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c/groups/fcd9246af06d4109a55ee05e6fc4437b/roles/88c855fc5a0a48368fb08bf18f344893` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.713069 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e21a6-fc83-47ac-9798-13f1732199a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f180df64cf54d3996588ba6518e4e4c, group_id=fcd9246af06d4109a55ee05e6fc4437b, role_id=88c855fc5a0a48368fb08bf18f344893)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:25.713069 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e21a6-fc83-47ac-9798-13f1732199a3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:25.727470 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 196/397] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:02:25 2022] PUT /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c/groups/fcd9246af06d4109a55ee05e6fc4437b/roles/88c855fc5a0a48368fb08bf18f344893 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:02:25.728817 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11127ee0-af80-408d-85ca-d500674b3b56 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.729131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11127ee0-af80-408d-85ca-d500674b3b56 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.729406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11127ee0-af80-408d-85ca-d500674b3b56 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.734871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 197/398] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:25.736241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ec7d8-1c6b-44f0-b501-cef0fc1d9a29 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.736526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ec7d8-1c6b-44f0-b501-cef0fc1d9a29 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.736801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ec7d8-1c6b-44f0-b501-cef0fc1d9a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.758764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-194ec7d8-1c6b-44f0-b501-cef0fc1d9a29 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:25.759953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-194ec7d8-1c6b-44f0-b501-cef0fc1d9a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:25.852250 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dec37c-41e0-4782-b913-835bf24d4826 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:25.883327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 198/399] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:25.912839 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:25.913596 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:25.913871 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:25.914141 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:25.914916 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:25.915222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:25.916704 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:25.927430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 202/400] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:25.933280 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ec7d8-1c6b-44f0-b501-cef0fc1d9a29 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:25.973424 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.049982 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ec7d8-1c6b-44f0-b501-cef0fc1d9a29 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.078041 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eea441-15b3-4938-a5dc-e31adbbafcb7 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.085081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfc801-d9d4-4035-9c97-442c2f64658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.085994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfc801-d9d4-4035-9c97-442c2f64658c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.086268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfc801-d9d4-4035-9c97-442c2f64658c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.086535 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfc801-d9d4-4035-9c97-442c2f64658c None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.087709 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfc801-d9d4-4035-9c97-442c2f64658c None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:26.089231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfc801-d9d4-4035-9c97-442c2f64658c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:26.091169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 199/401] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:25 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:26.096095 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfc801-d9d4-4035-9c97-442c2f64658c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.114387 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f0e4b-6249-4a41-8e3f-8f90d75902e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.115525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0e4b-6249-4a41-8e3f-8f90d75902e2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.116005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0e4b-6249-4a41-8e3f-8f90d75902e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.116466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f0e4b-6249-4a41-8e3f-8f90d75902e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.130102 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f0e4b-6249-4a41-8e3f-8f90d75902e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-916f0e4b-6249-4a41-8e3f-8f90d75902e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:26.141459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:26.142395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:26.143184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:26.145869 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 203/402] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:26.147676 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12591bad-ef1a-4e7f-a512-15416ae7557f None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.148106 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12591bad-ef1a-4e7f-a512-15416ae7557f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.148645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12591bad-ef1a-4e7f-a512-15416ae7557f None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.151026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 204/403] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:26.151859 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12591bad-ef1a-4e7f-a512-15416ae7557f None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.227305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfc801-d9d4-4035-9c97-442c2f64658c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.264649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12591bad-ef1a-4e7f-a512-15416ae7557f None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.269648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d857-b177-45e1-b053-64e46438ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.270533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d857-b177-45e1-b053-64e46438ae5e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.270804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d857-b177-45e1-b053-64e46438ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.271069 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d857-b177-45e1-b053-64e46438ae5e None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.272264 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d857-b177-45e1-b053-64e46438ae5e None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:26.274194 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d857-b177-45e1-b053-64e46438ae5e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:26.276880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 200/404] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:26 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:26.278560 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472702-b8b0-4271-ad93-039fc0ea405f None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.278836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472702-b8b0-4271-ad93-039fc0ea405f None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.279102 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472702-b8b0-4271-ad93-039fc0ea405f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.300185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-2f472702-b8b0-4271-ad93-039fc0ea405f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:26.301382 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-2f472702-b8b0-4271-ad93-039fc0ea405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:26.302667 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.303596 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.304082 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.304584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.305580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:26.306138 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:26.307747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:26.317000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 205/405] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:26.318169 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.388867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 201/406] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:26.390388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f472702-b8b0-4271-ad93-039fc0ea405f None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.428698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf78a7-02b9-4da0-afed-e81d728a5e80 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.463037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924bbf5-d395-473d-a689-2308c4f82939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.464265 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924bbf5-d395-473d-a689-2308c4f82939 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.464766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924bbf5-d395-473d-a689-2308c4f82939 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.465821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924bbf5-d395-473d-a689-2308c4f82939 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729/groups/fcd9246af06d4109a55ee05e6fc4437b/roles/67832478d7b8428d996975409814f845` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.468235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924bbf5-d395-473d-a689-2308c4f82939 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2350e443c204e6d9d190688211b1729, group_id=fcd9246af06d4109a55ee05e6fc4437b, role_id=67832478d7b8428d996975409814f845)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:26.469985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924bbf5-d395-473d-a689-2308c4f82939 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:26.489137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 206/407] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:02:26 2022] PUT /identity/v3/projects/a2350e443c204e6d9d190688211b1729/groups/fcd9246af06d4109a55ee05e6fc4437b/roles/67832478d7b8428d996975409814f845 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:26.490466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924bbf5-d395-473d-a689-2308c4f82939 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.531540 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f472702-b8b0-4271-ad93-039fc0ea405f None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.600343 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbddbc-4545-43e2-a52c-3d3fe10605d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.601187 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbddbc-4545-43e2-a52c-3d3fe10605d2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.601469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbddbc-4545-43e2-a52c-3d3fe10605d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.601744 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbddbc-4545-43e2-a52c-3d3fe10605d2 None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.602959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbddbc-4545-43e2-a52c-3d3fe10605d2 None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:26.604457 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbddbc-4545-43e2-a52c-3d3fe10605d2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:26.606296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 202/408] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:26 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:26.607512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7e0b-5867-44eb-9aa5-a1fb658ff85c None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.607837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7e0b-5867-44eb-9aa5-a1fb658ff85c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.608117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb7e0b-5867-44eb-9aa5-a1fb658ff85c None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.613653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 203/409] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:26.614552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7e0b-5867-44eb-9aa5-a1fb658ff85c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.615854 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924bbf5-d395-473d-a689-2308c4f82939 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.662508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14022ae-7d43-4014-8313-e2facd3fdbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.663858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14022ae-7d43-4014-8313-e2facd3fdbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.664792 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14022ae-7d43-4014-8313-e2facd3fdbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.665601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14022ae-7d43-4014-8313-e2facd3fdbf5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.681673 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14022ae-7d43-4014-8313-e2facd3fdbf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a14022ae-7d43-4014-8313-e2facd3fdbf5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:26.693383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:26.694576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:26.695566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:26.698471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 207/410] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:26.701208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d8cba-2cd4-413e-a112-13e8d9796576 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.701642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d8cba-2cd4-413e-a112-13e8d9796576 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.702090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d8cba-2cd4-413e-a112-13e8d9796576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.724811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-412d8cba-2cd4-413e-a112-13e8d9796576 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:26.726931 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-412d8cba-2cd4-413e-a112-13e8d9796576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:26.729902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb7e0b-5867-44eb-9aa5-a1fb658ff85c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.773288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf8e12-f3c6-4a21-bb3c-507de6518229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.774714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf8e12-f3c6-4a21-bb3c-507de6518229 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.774714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf8e12-f3c6-4a21-bb3c-507de6518229 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.774714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf8e12-f3c6-4a21-bb3c-507de6518229 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.775874 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf8e12-f3c6-4a21-bb3c-507de6518229 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:26.777348 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf8e12-f3c6-4a21-bb3c-507de6518229 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:26.779600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 204/411] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:26 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:26.780363 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf8e12-f3c6-4a21-bb3c-507de6518229 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.815715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 208/412] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:26.823249 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d8cba-2cd4-413e-a112-13e8d9796576 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.886564 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf8e12-f3c6-4a21-bb3c-507de6518229 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.923360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11937950-f697-431b-8635-5347bffd225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.923948 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11937950-f697-431b-8635-5347bffd225b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.924288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11937950-f697-431b-8635-5347bffd225b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.924784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11937950-f697-431b-8635-5347bffd225b None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.925344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11937950-f697-431b-8635-5347bffd225b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:26.925653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11937950-f697-431b-8635-5347bffd225b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:26.927016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11937950-f697-431b-8635-5347bffd225b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:26.936417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 205/413] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:26.942832 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11937950-f697-431b-8635-5347bffd225b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:26.944862 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d8cba-2cd4-413e-a112-13e8d9796576 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:26.979266 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1f190-1a6b-4910-ab94-c66c9b0fc26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:26.980493 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1f190-1a6b-4910-ab94-c66c9b0fc26c None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:26.981539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1f190-1a6b-4910-ab94-c66c9b0fc26c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:26.982136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1f190-1a6b-4910-ab94-c66c9b0fc26c None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729/groups/fcd9246af06d4109a55ee05e6fc4437b/roles/88c855fc5a0a48368fb08bf18f344893` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:26.984552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1f190-1a6b-4910-ab94-c66c9b0fc26c None admin] RBAC: Authorizing `identity:create_grant(project_id=a2350e443c204e6d9d190688211b1729, group_id=fcd9246af06d4109a55ee05e6fc4437b, role_id=88c855fc5a0a48368fb08bf18f344893)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:26.986243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1f190-1a6b-4910-ab94-c66c9b0fc26c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:27.013217 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 209/414] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:02:26 2022] PUT /identity/v3/projects/a2350e443c204e6d9d190688211b1729/groups/fcd9246af06d4109a55ee05e6fc4437b/roles/88c855fc5a0a48368fb08bf18f344893 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:27.014212 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1f190-1a6b-4910-ab94-c66c9b0fc26c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:27.057632 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11937950-f697-431b-8635-5347bffd225b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:27.119238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca82ed3-5af4-4761-b1ac-a2629f2364b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:27.120490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82ed3-5af4-4761-b1ac-a2629f2364b6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:27.122376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82ed3-5af4-4761-b1ac-a2629f2364b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:27.122908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca82ed3-5af4-4761-b1ac-a2629f2364b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:27.150928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca82ed3-5af4-4761-b1ac-a2629f2364b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-3ca82ed3-5af4-4761-b1ac-a2629f2364b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:27.170469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:27.171443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:27.172242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 206/415] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:27.192860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca82ed3-5af4-4761-b1ac-a2629f2364b6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:27.237476 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1f190-1a6b-4910-ab94-c66c9b0fc26c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:27.274064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152df0df-adb3-4453-942f-2a3298d2f08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:27.274962 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152df0df-adb3-4453-942f-2a3298d2f08e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:27.275242 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152df0df-adb3-4453-942f-2a3298d2f08e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:27.275514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152df0df-adb3-4453-942f-2a3298d2f08e None admin] PATH_INFO: `/v3/groups/a92cc2c5c08b4c3a97754cf4fafabf60` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:27.276723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152df0df-adb3-4453-942f-2a3298d2f08e None admin] RBAC: Authorizing `identity:get_group(group_id=a92cc2c5c08b4c3a97754cf4fafabf60)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:27.278745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152df0df-adb3-4453-942f-2a3298d2f08e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:27.280690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 210/416] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:27 2022] GET /identity/v3/groups/a92cc2c5c08b4c3a97754cf4fafabf60 => generated 229 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:27.286088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152df0df-adb3-4453-942f-2a3298d2f08e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:27.321989 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca82ed3-5af4-4761-b1ac-a2629f2364b6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:27.363482 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:27.364158 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:27.364804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:27.365115 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:27.366198 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:27.366513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:27.368353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:27.380228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 207/417] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:27.385620 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:27.393306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152df0df-adb3-4453-942f-2a3298d2f08e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:27.456580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e149c7c-5c46-46a9-bb1c-a11cc909ac7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:27.460533 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149c7c-5c46-46a9-bb1c-a11cc909ac7f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:27.460815 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149c7c-5c46-46a9-bb1c-a11cc909ac7f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:27.461167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e149c7c-5c46-46a9-bb1c-a11cc909ac7f None admin] PATH_INFO: `/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:27.462405 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e149c7c-5c46-46a9-bb1c-a11cc909ac7f None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51f62b70d42e281dc8a352a6f16d6)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:27.463855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e149c7c-5c46-46a9-bb1c-a11cc909ac7f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:27.467575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 211/418] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:27 2022] GET /identity/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:27.473301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e149c7c-5c46-46a9-bb1c-a11cc909ac7f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:27.552419 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1272fc6-334f-4985-ad40-3d9fefa81b5e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:27.586044 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dae77-0f15-4e90-9fc8-be51287eb925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:27.586689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dae77-0f15-4e90-9fc8-be51287eb925 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:27.587009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dae77-0f15-4e90-9fc8-be51287eb925 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:27.587281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dae77-0f15-4e90-9fc8-be51287eb925 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:27.655187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e149c7c-5c46-46a9-bb1c-a11cc909ac7f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:27.663523 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dae77-0f15-4e90-9fc8-be51287eb925 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:27.664976 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dae77-0f15-4e90-9fc8-be51287eb925 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:27.747749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d4b67-3b27-4b30-a3e1-ecc1b62bfe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:27.748622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d4b67-3b27-4b30-a3e1-ecc1b62bfe8e None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:27.748899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d4b67-3b27-4b30-a3e1-ecc1b62bfe8e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:27.749192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d4b67-3b27-4b30-a3e1-ecc1b62bfe8e None admin] PATH_INFO: `/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6/groups/a92cc2c5c08b4c3a97754cf4fafabf60/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:27.753433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d4b67-3b27-4b30-a3e1-ecc1b62bfe8e None admin] RBAC: Authorizing `identity:create_grant(project_id=f3f51f62b70d42e281dc8a352a6f16d6, group_id=a92cc2c5c08b4c3a97754cf4fafabf60, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:27.755162 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d4b67-3b27-4b30-a3e1-ecc1b62bfe8e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:27.778653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 212/419] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:02:27 2022] PUT /identity/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6/groups/a92cc2c5c08b4c3a97754cf4fafabf60/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:27.803593 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 208/420] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:27.809123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d4b67-3b27-4b30-a3e1-ecc1b62bfe8e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:27.925334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d4b67-3b27-4b30-a3e1-ecc1b62bfe8e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:27.964497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d128086-7d8a-42b2-a2e3-3c51decf26d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:27.966051 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d128086-7d8a-42b2-a2e3-3c51decf26d2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:27.966329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d128086-7d8a-42b2-a2e3-3c51decf26d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:27.966598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d128086-7d8a-42b2-a2e3-3c51decf26d2 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:27.967971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d128086-7d8a-42b2-a2e3-3c51decf26d2 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:27.969503 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d128086-7d8a-42b2-a2e3-3c51decf26d2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:27.971665 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 213/421] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:27 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:27.982687 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dae77-0f15-4e90-9fc8-be51287eb925 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:28.103039 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dae77-0f15-4e90-9fc8-be51287eb925 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:28.147887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bd548-4438-4b93-8873-a20cfcd62deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:28.149136 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bd548-4438-4b93-8873-a20cfcd62deb None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:28.149614 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bd548-4438-4b93-8873-a20cfcd62deb None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:28.150105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bd548-4438-4b93-8873-a20cfcd62deb None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:28.152074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bd548-4438-4b93-8873-a20cfcd62deb None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:28.153864 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bd548-4438-4b93-8873-a20cfcd62deb None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:28.156046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 209/422] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:27 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:28.161630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d128086-7d8a-42b2-a2e3-3c51decf26d2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:28.316234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d128086-7d8a-42b2-a2e3-3c51decf26d2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:28.362221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3fc19-c846-4ac2-a06f-7111821493bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:28.362808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fc19-c846-4ac2-a06f-7111821493bb None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:28.363089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fc19-c846-4ac2-a06f-7111821493bb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:28.363365 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3fc19-c846-4ac2-a06f-7111821493bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:28.364205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3fc19-c846-4ac2-a06f-7111821493bb None admin] RBAC: Adding query filter params (group.id=fcd9246af06d4109a55ee05e6fc4437b, role.id=67832478d7b8428d996975409814f845, scope.project.id=8f180df64cf54d3996588ba6518e4e4c) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:28.364517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3fc19-c846-4ac2-a06f-7111821493bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:28.365992 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3fc19-c846-4ac2-a06f-7111821493bb None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:28.377717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 214/423] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:28 2022] GET /identity/v3/role_assignments?group.id=fcd9246af06d4109a55ee05e6fc4437b&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=67832478d7b8428d996975409814f845 => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:28.469154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f76d0-05e8-4176-b22f-d300922b1ca5 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:28.469810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f76d0-05e8-4176-b22f-d300922b1ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:28.470284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f76d0-05e8-4176-b22f-d300922b1ca5 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:28.471867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 210/424] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:28.477788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb7bdb-eaeb-4e4e-87e2-47b7b83833ea None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:28.478089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb7bdb-eaeb-4e4e-87e2-47b7b83833ea None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:28.478351 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb7bdb-eaeb-4e4e-87e2-47b7b83833ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:28.502730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-e7cb7bdb-eaeb-4e4e-87e2-47b7b83833ea None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:28.503945 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-e7cb7bdb-eaeb-4e4e-87e2-47b7b83833ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:28.598615 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 215/425] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:28.647194 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f76d0-05e8-4176-b22f-d300922b1ca5 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:28.779256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f76d0-05e8-4176-b22f-d300922b1ca5 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:28.823108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebb0d6-5afc-4543-8673-ce9f05f9f482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:28.823931 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebb0d6-5afc-4543-8673-ce9f05f9f482 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:28.824209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebb0d6-5afc-4543-8673-ce9f05f9f482 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:28.825067 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebb0d6-5afc-4543-8673-ce9f05f9f482 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:28.836087 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebb0d6-5afc-4543-8673-ce9f05f9f482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-18ebb0d6-5afc-4543-8673-ce9f05f9f482 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:28.844502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:28.845444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:28.846284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 211/426] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:28.851553 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb7bdb-eaeb-4e4e-87e2-47b7b83833ea None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:28.967400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb7bdb-eaeb-4e4e-87e2-47b7b83833ea None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:29.010770 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:29.011455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.011738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.012616 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.013406 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:29.013709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:29.015042 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590447e-a0e4-49e9-8a54-f1b31702520e None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.015480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:29.015969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590447e-a0e4-49e9-8a54-f1b31702520e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.016440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590447e-a0e4-49e9-8a54-f1b31702520e None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.018485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 212/427] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:29.024479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 216/428] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:29.025982 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc6fdd-30b4-4158-8f1b-4655d6a3887c None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.026265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc6fdd-30b4-4158-8f1b-4655d6a3887c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.026552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc6fdd-30b4-4158-8f1b-4655d6a3887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.030322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:29.049682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-a4cc6fdd-30b4-4158-8f1b-4655d6a3887c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:29.051450 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-a4cc6fdd-30b4-4158-8f1b-4655d6a3887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:29.143113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 213/429] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:29.159346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7cfd5-1dc4-478a-9540-2cc0083d467b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:29.178126 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc6fdd-30b4-4158-8f1b-4655d6a3887c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:29.201602 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1f4ff-c278-4137-89d1-93044180c390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:29.202679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f4ff-c278-4137-89d1-93044180c390 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.202957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f4ff-c278-4137-89d1-93044180c390 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.203227 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f4ff-c278-4137-89d1-93044180c390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.286509 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1f4ff-c278-4137-89d1-93044180c390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:29.287897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1f4ff-c278-4137-89d1-93044180c390 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:29.305518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc6fdd-30b4-4158-8f1b-4655d6a3887c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:29.343628 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448facc-1d8a-4b9a-b1af-f96a0a67a905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:29.344469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448facc-1d8a-4b9a-b1af-f96a0a67a905 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.344745 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448facc-1d8a-4b9a-b1af-f96a0a67a905 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.345013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448facc-1d8a-4b9a-b1af-f96a0a67a905 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.355159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448facc-1d8a-4b9a-b1af-f96a0a67a905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f448facc-1d8a-4b9a-b1af-f96a0a67a905 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:29.365411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:29.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:29.367129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 214/430] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:29.374349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448facc-1d8a-4b9a-b1af-f96a0a67a905 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:29.399428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 217/431] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:29.400531 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1f4ff-c278-4137-89d1-93044180c390 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:29.477838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448facc-1d8a-4b9a-b1af-f96a0a67a905 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:29.516015 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:29.516601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1f4ff-c278-4137-89d1-93044180c390 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:29.516797 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.516952 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.517244 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.518005 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:29.518305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:29.519683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:29.530723 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 215/432] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:02:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:29.536693 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:29.564320 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4701d-6474-43b4-bb2a-a2c9db38c5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:29.565213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4701d-6474-43b4-bb2a-a2c9db38c5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.565493 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4701d-6474-43b4-bb2a-a2c9db38c5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.565767 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4701d-6474-43b4-bb2a-a2c9db38c5b0 None admin] PATH_INFO: `/v3/projects/8f180df64cf54d3996588ba6518e4e4c` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.566960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4701d-6474-43b4-bb2a-a2c9db38c5b0 None admin] RBAC: Authorizing `identity:get_project(project_id=8f180df64cf54d3996588ba6518e4e4c)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:29.568425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4701d-6474-43b4-bb2a-a2c9db38c5b0 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:29.575422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 218/433] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:29 2022] GET /identity/v3/projects/8f180df64cf54d3996588ba6518e4e4c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:29.576264 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4701d-6474-43b4-bb2a-a2c9db38c5b0 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:29.662602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535b8bb-36d0-4b3b-a08c-0408af1dd0fe None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:29.677184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4701d-6474-43b4-bb2a-a2c9db38c5b0 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:29.704192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65318602-0e42-47a2-998c-3545e8509610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:29.705444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318602-0e42-47a2-998c-3545e8509610 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.705729 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318602-0e42-47a2-998c-3545e8509610 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.706004 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65318602-0e42-47a2-998c-3545e8509610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.712156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b123d04-48e4-4e77-9b17-d72e2b328eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:29.713182 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b123d04-48e4-4e77-9b17-d72e2b328eaa None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.713564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b123d04-48e4-4e77-9b17-d72e2b328eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.713938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b123d04-48e4-4e77-9b17-d72e2b328eaa None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.715345 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b123d04-48e4-4e77-9b17-d72e2b328eaa None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:29.716783 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b123d04-48e4-4e77-9b17-d72e2b328eaa None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:29.724987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 219/434] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:29 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:29.726278 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ce941-335f-4fe8-9e88-81664fce6b52 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.726278 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ce941-335f-4fe8-9e88-81664fce6b52 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.726278 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ce941-335f-4fe8-9e88-81664fce6b52 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.728076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 220/435] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:29.728909 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ce941-335f-4fe8-9e88-81664fce6b52 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:29.779280 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65318602-0e42-47a2-998c-3545e8509610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:29.780934 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65318602-0e42-47a2-998c-3545e8509610 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:29.828808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ce941-335f-4fe8-9e88-81664fce6b52 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:29.861672 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28ba8f-6109-44ba-aa55-55f58fd03f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:29.862283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ba8f-6109-44ba-aa55-55f58fd03f51 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.862563 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ba8f-6109-44ba-aa55-55f58fd03f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.862836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ba8f-6109-44ba-aa55-55f58fd03f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.863666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28ba8f-6109-44ba-aa55-55f58fd03f51 None admin] RBAC: Adding query filter params (group.id=fcd9246af06d4109a55ee05e6fc4437b, role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=8f180df64cf54d3996588ba6518e4e4c) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:29.863977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28ba8f-6109-44ba-aa55-55f58fd03f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:29.865439 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28ba8f-6109-44ba-aa55-55f58fd03f51 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:29.872809 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 221/436] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:29 2022] GET /identity/v3/role_assignments?group.id=fcd9246af06d4109a55ee05e6fc4437b&scope.project.id=8f180df64cf54d3996588ba6518e4e4c&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:29.874249 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a8b77-3c92-43e0-8591-f4e912cecc54 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:29.874670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a8b77-3c92-43e0-8591-f4e912cecc54 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:29.875100 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a8b77-3c92-43e0-8591-f4e912cecc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:29.888663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 216/437] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:02:29.889838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65318602-0e42-47a2-998c-3545e8509610 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:29.895054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-e97a8b77-3c92-43e0-8591-f4e912cecc54 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:29.896525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-e97a8b77-3c92-43e0-8591-f4e912cecc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:29.988273 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 222/438] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:30.007798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65318602-0e42-47a2-998c-3545e8509610 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.013970 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a8b77-3c92-43e0-8591-f4e912cecc54 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.059168 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb2fbb-2a6f-4f04-9961-79b1ce590eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:30.060722 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb2fbb-2a6f-4f04-9961-79b1ce590eef None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.060996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb2fbb-2a6f-4f04-9961-79b1ce590eef None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.061313 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb2fbb-2a6f-4f04-9961-79b1ce590eef None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.063023 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb2fbb-2a6f-4f04-9961-79b1ce590eef None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:30.064464 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb2fbb-2a6f-4f04-9961-79b1ce590eef None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:30.067038 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 217/439] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:29 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:30.072410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb2fbb-2a6f-4f04-9961-79b1ce590eef None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.148405 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a8b77-3c92-43e0-8591-f4e912cecc54 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.191301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3635b42-0774-4ed4-9a65-50de13220716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:30.192278 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3635b42-0774-4ed4-9a65-50de13220716 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.192845 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3635b42-0774-4ed4-9a65-50de13220716 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.192845 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3635b42-0774-4ed4-9a65-50de13220716 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.195562 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb2fbb-2a6f-4f04-9961-79b1ce590eef None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.205110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3635b42-0774-4ed4-9a65-50de13220716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a3635b42-0774-4ed4-9a65-50de13220716 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:30.213669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:30.214647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:30.215480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:30.219644 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 223/440] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:30.220873 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3635b42-0774-4ed4-9a65-50de13220716 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.242138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b536ac2-d116-4b16-88d4-17a8fb8dfad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:30.243364 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b536ac2-d116-4b16-88d4-17a8fb8dfad1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.243643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b536ac2-d116-4b16-88d4-17a8fb8dfad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.243912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b536ac2-d116-4b16-88d4-17a8fb8dfad1 None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.246000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b536ac2-d116-4b16-88d4-17a8fb8dfad1 None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:30.247610 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b536ac2-d116-4b16-88d4-17a8fb8dfad1 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:30.254977 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 218/441] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:30.255812 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b536ac2-d116-4b16-88d4-17a8fb8dfad1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.335652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3635b42-0774-4ed4-9a65-50de13220716 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.371716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b536ac2-d116-4b16-88d4-17a8fb8dfad1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.383189 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:30.383950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.384059 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.384328 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.385194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:30.385504 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:30.386960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:30.396943 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 224/442] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:30.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.415587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15bb9f-f26b-4988-9aee-2cd4e566db5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:30.416257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15bb9f-f26b-4988-9aee-2cd4e566db5c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.416604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15bb9f-f26b-4988-9aee-2cd4e566db5c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.416878 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15bb9f-f26b-4988-9aee-2cd4e566db5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.417801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15bb9f-f26b-4988-9aee-2cd4e566db5c None admin] RBAC: Adding query filter params (group.id=fcd9246af06d4109a55ee05e6fc4437b, role.id=67832478d7b8428d996975409814f845, scope.project.id=a2350e443c204e6d9d190688211b1729) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:30.418764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15bb9f-f26b-4988-9aee-2cd4e566db5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:30.420267 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15bb9f-f26b-4988-9aee-2cd4e566db5c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:30.426439 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 219/443] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/role_assignments?group.id=fcd9246af06d4109a55ee05e6fc4437b&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=67832478d7b8428d996975409814f845 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:30.508891 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fb5bf-0902-4fe0-bf58-4608efff781f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.543685 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cadd720-9613-4339-87b2-4c1ec6f1d754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:30.544337 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadd720-9613-4339-87b2-4c1ec6f1d754 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.544614 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadd720-9613-4339-87b2-4c1ec6f1d754 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.544886 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cadd720-9613-4339-87b2-4c1ec6f1d754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.587866 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fb1d7-2f56-4b3c-82bc-cb157a819126 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.588240 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fb1d7-2f56-4b3c-82bc-cb157a819126 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.588404 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fb1d7-2f56-4b3c-82bc-cb157a819126 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.590054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 220/444] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:02:30.595325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398761c-94fc-4fed-9618-a5587bb7d593 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.595603 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398761c-94fc-4fed-9618-a5587bb7d593 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398761c-94fc-4fed-9618-a5587bb7d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.613082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-f398761c-94fc-4fed-9618-a5587bb7d593 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:30.614063 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-f398761c-94fc-4fed-9618-a5587bb7d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:30.617854 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cadd720-9613-4339-87b2-4c1ec6f1d754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:30.619321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cadd720-9613-4339-87b2-4c1ec6f1d754 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:30.689526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 221/445] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:30.712357 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398761c-94fc-4fed-9618-a5587bb7d593 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.727082 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 225/446] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:02:30.733926 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cadd720-9613-4339-87b2-4c1ec6f1d754 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.825478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398761c-94fc-4fed-9618-a5587bb7d593 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.846736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cadd720-9613-4339-87b2-4c1ec6f1d754 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.858533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3590c-88e3-4fa1-9fce-d45d7d081aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:30.859326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3590c-88e3-4fa1-9fce-d45d7d081aac None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.859604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3590c-88e3-4fa1-9fce-d45d7d081aac None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.859881 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3590c-88e3-4fa1-9fce-d45d7d081aac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.869440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3590c-88e3-4fa1-9fce-d45d7d081aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-3ca3590c-88e3-4fa1-9fce-d45d7d081aac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:30.877024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:30.877965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:30.878774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:30.879379 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8058ab1-fe33-44ec-aff5-59141e893a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:30.879465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 222/447] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:30.879798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8058ab1-fe33-44ec-aff5-59141e893a57 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:30.879798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8058ab1-fe33-44ec-aff5-59141e893a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:30.879798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8058ab1-fe33-44ec-aff5-59141e893a57 None admin] PATH_INFO: `/v3/projects/a2350e443c204e6d9d190688211b1729` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:30.880591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8058ab1-fe33-44ec-aff5-59141e893a57 None admin] RBAC: Authorizing `identity:get_project(project_id=a2350e443c204e6d9d190688211b1729)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:30.882003 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8058ab1-fe33-44ec-aff5-59141e893a57 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:30.888494 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 226/448] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/projects/a2350e443c204e6d9d190688211b1729 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:30.888628 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3590c-88e3-4fa1-9fce-d45d7d081aac None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.889416 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8058ab1-fe33-44ec-aff5-59141e893a57 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:30.976464 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3590c-88e3-4fa1-9fce-d45d7d081aac None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:30.979982 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8058ab1-fe33-44ec-aff5-59141e893a57 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:31.007295 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:31.007949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:31.008226 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:31.008498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:31.009260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:31.009561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:31.010910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:31.012440 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2308b10-fe51-4694-8264-6256c9994826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:31.013376 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2308b10-fe51-4694-8264-6256c9994826 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:31.013652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2308b10-fe51-4694-8264-6256c9994826 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:31.013918 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2308b10-fe51-4694-8264-6256c9994826 None admin] PATH_INFO: `/v3/groups/fcd9246af06d4109a55ee05e6fc4437b` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:31.015097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2308b10-fe51-4694-8264-6256c9994826 None admin] RBAC: Authorizing `identity:get_group(group_id=fcd9246af06d4109a55ee05e6fc4437b)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:31.016497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2308b10-fe51-4694-8264-6256c9994826 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:31.018139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 227/449] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/groups/fcd9246af06d4109a55ee05e6fc4437b => generated 226 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:31.019159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 223/450] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:31.023733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2308b10-fe51-4694-8264-6256c9994826 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:31.024529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:31.118699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2308b10-fe51-4694-8264-6256c9994826 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:31.125209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d8220-d20e-4d52-9c3e-b47fb837c418 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:31.165924 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ede1ac-eb1f-4cf2-92fb-b4d2c45eb9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:31.166194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:31.166570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede1ac-eb1f-4cf2-92fb-b4d2c45eb9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:31.166827 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:31.166915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede1ac-eb1f-4cf2-92fb-b4d2c45eb9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:31.167104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:31.167197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede1ac-eb1f-4cf2-92fb-b4d2c45eb9f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:31.167375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:31.169240 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] RBAC: Adding query filter params (group.id=fcd9246af06d4109a55ee05e6fc4437b, role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=a2350e443c204e6d9d190688211b1729) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:31.169703 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:31.171929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:31.178947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 228/451] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:31 2022] GET /identity/v3/role_assignments?group.id=fcd9246af06d4109a55ee05e6fc4437b&scope.project.id=a2350e443c204e6d9d190688211b1729&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:31.243349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ede1ac-eb1f-4cf2-92fb-b4d2c45eb9f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:31.244737 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ede1ac-eb1f-4cf2-92fb-b4d2c45eb9f0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:31.344834 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 224/452] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:02:31.350514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:31.448463 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58841c63-93bc-42eb-98f7-6669440c0f4d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:31.481684 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaab0f3-2bec-4dd9-8832-28580817aea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:31.482538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaab0f3-2bec-4dd9-8832-28580817aea0 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:31.482821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaab0f3-2bec-4dd9-8832-28580817aea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:31.483093 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaab0f3-2bec-4dd9-8832-28580817aea0 None admin] PATH_INFO: `/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:31.484277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaab0f3-2bec-4dd9-8832-28580817aea0 None admin] RBAC: Authorizing `identity:get_project(project_id=f3f51f62b70d42e281dc8a352a6f16d6)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:31.485786 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaab0f3-2bec-4dd9-8832-28580817aea0 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:31.487657 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 229/453] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:02:31 2022] GET /identity/v3/projects/f3f51f62b70d42e281dc8a352a6f16d6 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:31.493457 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ede1ac-eb1f-4cf2-92fb-b4d2c45eb9f0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:31.595415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ede1ac-eb1f-4cf2-92fb-b4d2c45eb9f0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:31.626980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d5142-144e-4536-8248-6ff468566099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:31.627809 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d5142-144e-4536-8248-6ff468566099 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:31.628084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d5142-144e-4536-8248-6ff468566099 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:31.628353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d5142-144e-4536-8248-6ff468566099 None admin] PATH_INFO: `/v3/groups/a92cc2c5c08b4c3a97754cf4fafabf60` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:31.629760 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d5142-144e-4536-8248-6ff468566099 None admin] RBAC: Authorizing `identity:get_group(group_id=a92cc2c5c08b4c3a97754cf4fafabf60)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:31.631188 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d5142-144e-4536-8248-6ff468566099 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:31.632765 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 225/454] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:02:31 2022] GET /identity/v3/groups/a92cc2c5c08b4c3a97754cf4fafabf60 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:31.639091 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaab0f3-2bec-4dd9-8832-28580817aea0 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:31.731942 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaab0f3-2bec-4dd9-8832-28580817aea0 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:31.764276 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d64b1-23bd-4365-9d87-989f24cad41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:31.764850 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d64b1-23bd-4365-9d87-989f24cad41f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:31.765159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d64b1-23bd-4365-9d87-989f24cad41f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:31.765433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d64b1-23bd-4365-9d87-989f24cad41f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:31.766259 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d64b1-23bd-4365-9d87-989f24cad41f None admin] RBAC: Adding query filter params (group.id=a92cc2c5c08b4c3a97754cf4fafabf60, role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=f3f51f62b70d42e281dc8a352a6f16d6) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:31.766565 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d64b1-23bd-4365-9d87-989f24cad41f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:31.768014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d64b1-23bd-4365-9d87-989f24cad41f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:31.773510 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 230/455] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:02:31 2022] GET /identity/v3/role_assignments?group.id=a92cc2c5c08b4c3a97754cf4fafabf60&scope.project.id=f3f51f62b70d42e281dc8a352a6f16d6&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 611 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:34.553429 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6be7-97c2-43bd-8a2d-e295aada30e7 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:34.554473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6be7-97c2-43bd-8a2d-e295aada30e7 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:34.554959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6be7-97c2-43bd-8a2d-e295aada30e7 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:34.556769 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 226/456] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:34.561878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f11b-87c2-41ab-b69c-dcf152a56fa2 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:34.562682 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f11b-87c2-41ab-b69c-dcf152a56fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:34.563168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f11b-87c2-41ab-b69c-dcf152a56fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:34.584994 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-e980f11b-87c2-41ab-b69c-dcf152a56fa2 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:34.584994 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-e980f11b-87c2-41ab-b69c-dcf152a56fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:34.608969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230b3bb-20ab-4519-8a08-f81df1b1f499 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:34.609613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230b3bb-20ab-4519-8a08-f81df1b1f499 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:34.610376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230b3bb-20ab-4519-8a08-f81df1b1f499 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:34.615542 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 227/457] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:34.619610 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149dd1b-4078-406c-9ff8-4477b5f49794 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:34.620269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149dd1b-4078-406c-9ff8-4477b5f49794 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:34.621127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149dd1b-4078-406c-9ff8-4477b5f49794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:34.654743 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-9149dd1b-4078-406c-9ff8-4477b5f49794 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:34.658540 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-9149dd1b-4078-406c-9ff8-4477b5f49794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:34.701007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 231/458] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:34.725292 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f11b-87c2-41ab-b69c-dcf152a56fa2 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:34.769393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 228/459] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:34.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149dd1b-4078-406c-9ff8-4477b5f49794 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:34.884025 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f11b-87c2-41ab-b69c-dcf152a56fa2 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:34.926468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d50d2-a9cb-4c1c-b6bc-49efb6bcfe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:34.927895 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d50d2-a9cb-4c1c-b6bc-49efb6bcfe62 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:34.928586 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d50d2-a9cb-4c1c-b6bc-49efb6bcfe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:34.929013 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d50d2-a9cb-4c1c-b6bc-49efb6bcfe62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:34.942674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d50d2-a9cb-4c1c-b6bc-49efb6bcfe62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:34.944459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d50d2-a9cb-4c1c-b6bc-49efb6bcfe62 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:34.944574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149dd1b-4078-406c-9ff8-4477b5f49794 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-432d50d2-a9cb-4c1c-b6bc-49efb6bcfe62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:34.967643 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:34.968754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:34.969838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:34.969838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:34.969838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:34.969838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:34.977080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 232/460] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:34.977080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28f3e3-9d95-4656-a2cb-3d1447c4c18a None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:34.977080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28f3e3-9d95-4656-a2cb-3d1447c4c18a None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:34.977080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28f3e3-9d95-4656-a2cb-3d1447c4c18a None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:34.977080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 233/461] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:34.977472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28f3e3-9d95-4656-a2cb-3d1447c4c18a None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:34.990971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf57223-8695-4843-aafe-50f0c08c3b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:34.991761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57223-8695-4843-aafe-50f0c08c3b10 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:34.992040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57223-8695-4843-aafe-50f0c08c3b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:34.992311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57223-8695-4843-aafe-50f0c08c3b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.004262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf57223-8695-4843-aafe-50f0c08c3b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.006283 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf57223-8695-4843-aafe-50f0c08c3b10 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-5cf57223-8695-4843-aafe-50f0c08c3b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:35.023302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:35.024182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:35.025029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:35.025029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:35.025029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:35.025029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:35.027290 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 229/462] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:35.029261 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8becc04-2531-4684-824a-0b6800b659d5 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.029722 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8becc04-2531-4684-824a-0b6800b659d5 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.030168 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8becc04-2531-4684-824a-0b6800b659d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.046837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-f8becc04-2531-4684-824a-0b6800b659d5 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:35.048247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-f8becc04-2531-4684-824a-0b6800b659d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:35.106769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28f3e3-9d95-4656-a2cb-3d1447c4c18a None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:35.120106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 230/463] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:35.121798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8becc04-2531-4684-824a-0b6800b659d5 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:35.146532 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:35.147268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.147638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.147983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.149263 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:35.149646 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.151503 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.177122 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 234/464] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:35.177122 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:35.270581 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8becc04-2531-4684-824a-0b6800b659d5 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:35.281962 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be12ca-2f72-4c0a-bd61-76f628f79fba None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:35.304388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:35.305315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.305777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.306247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.307165 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:35.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.309838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.317345 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:35.318420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.318875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.319319 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.323021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 231/465] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:35.324053 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:35.330889 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.332668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:35.351683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:35.352564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:35.353471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:35.353471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:35.353471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:35.353471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:35.360408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 235/466] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:35.360408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:35.424487 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf5224-0594-492a-9642-ea1bd4f696e8 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:35.457181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c3cfb-1335-434d-9f4b-b68068c7f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:35.457857 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c3cfb-1335-434d-9f4b-b68068c7f2cf None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.458137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c3cfb-1335-434d-9f4b-b68068c7f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.458409 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c3cfb-1335-434d-9f4b-b68068c7f2cf None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.459109 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c3cfb-1335-434d-9f4b-b68068c7f2cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.460529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c3cfb-1335-434d-9f4b-b68068c7f2cf None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.461426 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c70b58-315e-419b-8c86-059437baac9c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:35.481664 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 232/467] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:35 2022] POST /identity/v3/users => generated 248 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:35.483332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7061e-1c1c-4b5f-bb49-d861c1d52cf2 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.483767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7061e-1c1c-4b5f-bb49-d861c1d52cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.484202 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7061e-1c1c-4b5f-bb49-d861c1d52cf2 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.485547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 233/468] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:35.486284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7061e-1c1c-4b5f-bb49-d861c1d52cf2 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:35.517864 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b54a3-b383-4cc0-bd1e-db7d2d3214c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:35.518590 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b54a3-b383-4cc0-bd1e-db7d2d3214c3 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.519126 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b54a3-b383-4cc0-bd1e-db7d2d3214c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.519465 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b54a3-b383-4cc0-bd1e-db7d2d3214c3 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.520199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b54a3-b383-4cc0-bd1e-db7d2d3214c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.522426 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b54a3-b383-4cc0-bd1e-db7d2d3214c3 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.549713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 236/469] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:35 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:35.552593 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a30ab-b983-408d-8b16-28b124a6f33a None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.553268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a30ab-b983-408d-8b16-28b124a6f33a None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.553762 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a30ab-b983-408d-8b16-28b124a6f33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.590837 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-0e0a30ab-b983-408d-8b16-28b124a6f33a None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:35.593847 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-0e0a30ab-b983-408d-8b16-28b124a6f33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:35.691750 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7061e-1c1c-4b5f-bb49-d861c1d52cf2 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:35.714129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 237/470] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:35.738275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:35.739330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.739808 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.740350 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.741583 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a30ab-b983-408d-8b16-28b124a6f33a None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:35.741949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:35.742269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.743686 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.765802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 234/471] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:35.771350 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:35.869658 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a30ab-b983-408d-8b16-28b124a6f33a None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:35.888969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea581e-bb99-44e3-88ce-40376ff44930 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:35.911226 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe197807-9c57-4e69-9ec6-b010ebdfeae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:35.912015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197807-9c57-4e69-9ec6-b010ebdfeae3 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.912289 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197807-9c57-4e69-9ec6-b010ebdfeae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.912558 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe197807-9c57-4e69-9ec6-b010ebdfeae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.923049 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe197807-9c57-4e69-9ec6-b010ebdfeae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.924508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe197807-9c57-4e69-9ec6-b010ebdfeae3 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.929027 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebaf9d-b137-44da-80d9-32287f70be48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:35.929724 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebaf9d-b137-44da-80d9-32287f70be48 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.930004 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebaf9d-b137-44da-80d9-32287f70be48 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.930279 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebaf9d-b137-44da-80d9-32287f70be48 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.930979 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebaf9d-b137-44da-80d9-32287f70be48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:35.932402 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebaf9d-b137-44da-80d9-32287f70be48 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-fe197807-9c57-4e69-9ec6-b010ebdfeae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:35.942484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:35.943467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:35.944325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:35.944325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:35.944325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:35.944325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:35.944325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 238/472] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:35.945023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586f29b-51ca-43fe-b1d4-455002d416ad None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.945294 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586f29b-51ca-43fe-b1d4-455002d416ad None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.945450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586f29b-51ca-43fe-b1d4-455002d416ad None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.947980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 239/473] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:35.952326 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e528c2-d1ac-4a3f-ba96-c6c1fe6675c6 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:35.952598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e528c2-d1ac-4a3f-ba96-c6c1fe6675c6 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:35.952865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e528c2-d1ac-4a3f-ba96-c6c1fe6675c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:35.962917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 235/474] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:35 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:35.963799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebaf9d-b137-44da-80d9-32287f70be48 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:35.973601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-a4e528c2-d1ac-4a3f-ba96-c6c1fe6675c6 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:35.974776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-a4e528c2-d1ac-4a3f-ba96-c6c1fe6675c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:36.092570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 240/475] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:36.142535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e528c2-d1ac-4a3f-ba96-c6c1fe6675c6 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:36.154862 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebaf9d-b137-44da-80d9-32287f70be48 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:36.194326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:36.194928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:36.195207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:36.201129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:36.201129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:36.201129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:36.201129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:36.215433 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 236/476] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:36.220516 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:36.317506 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e528c2-d1ac-4a3f-ba96-c6c1fe6675c6 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:36.326322 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d9ce0-436f-48d2-9b2a-478b5165bb25 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:36.367702 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698b743-7fa0-4a13-a82f-0b8f9062f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:36.367931 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:36.368370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698b743-7fa0-4a13-a82f-0b8f9062f5d2 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:36.368477 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:36.368636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698b743-7fa0-4a13-a82f-0b8f9062f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:36.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:36.368905 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698b743-7fa0-4a13-a82f-0b8f9062f5d2 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:36.369000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:36.369657 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698b743-7fa0-4a13-a82f-0b8f9062f5d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:36.369760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:36.371621 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698b743-7fa0-4a13-a82f-0b8f9062f5d2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:36.371744 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:36.380136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.sql.core [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:02:36.380136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 15:02:36.380136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [parameters: {'id': '3c55104015ee44f1bcd0d7f93ec3a8ff', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 15:02:36.380136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80062) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:02:36.397497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 237/477] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:36 2022] POST /identity/v3/users => generated 249 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application conn.query(q) May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application result.read() May 09 15:02:36.404519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return ref.to_dict() May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:02:36.405425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application next(self.gen) May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application yield resource May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application next(self.gen) May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise self.value May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application session.commit() May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:02:36.406346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.session.flush() May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application compat.raise_( May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:02:36.407214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:02:36.408107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application conn.query(q) May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application result.read() May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:02:36.408934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [parameters: {'id': '3c55104015ee44f1bcd0d7f93ec3a8ff', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:36.411707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 15:02:36.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:36.421970 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 241/478] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:02:36 2022] POST /identity/v3/roles => generated 144 bytes in 264 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:36.422107 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698b743-7fa0-4a13-a82f-0b8f9062f5d2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:36.518279 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698b743-7fa0-4a13-a82f-0b8f9062f5d2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:36.560349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d2af8-517e-4e84-9ebf-b0d5fdf125e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:36.561200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d2af8-517e-4e84-9ebf-b0d5fdf125e8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:36.561479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d2af8-517e-4e84-9ebf-b0d5fdf125e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:36.561793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d2af8-517e-4e84-9ebf-b0d5fdf125e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:36.579651 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d2af8-517e-4e84-9ebf-b0d5fdf125e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e46d2af8-517e-4e84-9ebf-b0d5fdf125e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:36.588007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:36.590338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:02:36.592265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:36.593771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 238/479] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:02:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:36.599483 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:36.742191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453f9af-1dea-495c-8630-80a5c8d06c4e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:36.791798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c221e-2a14-4e79-bc29-19b73ac1f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:36.792389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c221e-2a14-4e79-bc29-19b73ac1f03b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:36.792665 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c221e-2a14-4e79-bc29-19b73ac1f03b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:36.792958 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c221e-2a14-4e79-bc29-19b73ac1f03b None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:36.793739 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c221e-2a14-4e79-bc29-19b73ac1f03b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:36.794037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c221e-2a14-4e79-bc29-19b73ac1f03b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:36.795419 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c221e-2a14-4e79-bc29-19b73ac1f03b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:36.807320 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 242/480] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:38.307384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368f6ca-4306-4034-bfea-d4f541a5b82a None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:38.308081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368f6ca-4306-4034-bfea-d4f541a5b82a None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:38.308081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368f6ca-4306-4034-bfea-d4f541a5b82a None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:38.310165 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 239/481] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:38.315378 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe28be5-e32d-4726-98a3-b13b5e8cd0e0 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:38.315881 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe28be5-e32d-4726-98a3-b13b5e8cd0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:38.316037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe28be5-e32d-4726-98a3-b13b5e8cd0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:38.338021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-9fe28be5-e32d-4726-98a3-b13b5e8cd0e0 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:38.339215 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-9fe28be5-e32d-4726-98a3-b13b5e8cd0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:38.379822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db57e-4766-4466-87c3-c51b3532228b None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:38.380114 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db57e-4766-4466-87c3-c51b3532228b None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:38.380272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4db57e-4766-4466-87c3-c51b3532228b None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:38.381872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 240/482] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:38.387784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f47779-d098-471e-ae55-f8e2cc9185fb None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:38.388065 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f47779-d098-471e-ae55-f8e2cc9185fb None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:38.388339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f47779-d098-471e-ae55-f8e2cc9185fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:38.413973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-31f47779-d098-471e-ae55-f8e2cc9185fb None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:38.415160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-31f47779-d098-471e-ae55-f8e2cc9185fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:38.418429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 243/483] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:38.484695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe28be5-e32d-4726-98a3-b13b5e8cd0e0 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:38.500177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 241/484] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:38.536045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f47779-d098-471e-ae55-f8e2cc9185fb None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:38.618933 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe28be5-e32d-4726-98a3-b13b5e8cd0e0 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:38.645598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f47779-d098-471e-ae55-f8e2cc9185fb None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:38.655059 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e114e7-6c14-49f0-9c3a-b11eb3cd5bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:38.656147 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e114e7-6c14-49f0-9c3a-b11eb3cd5bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:38.656630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e114e7-6c14-49f0-9c3a-b11eb3cd5bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:38.657142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e114e7-6c14-49f0-9c3a-b11eb3cd5bb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:38.667687 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e114e7-6c14-49f0-9c3a-b11eb3cd5bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-26e114e7-6c14-49f0-9c3a-b11eb3cd5bb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:38.675715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:38.676645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:38.677499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:38.679422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed3ac9-0993-4157-a7f7-277d552debd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:38.680313 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed3ac9-0993-4157-a7f7-277d552debd1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:38.680486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed3ac9-0993-4157-a7f7-277d552debd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:38.680686 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 244/485] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:38.680790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed3ac9-0993-4157-a7f7-277d552debd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:38.687021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e114e7-6c14-49f0-9c3a-b11eb3cd5bb2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:38.691707 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed3ac9-0993-4157-a7f7-277d552debd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f8ed3ac9-0993-4157-a7f7-277d552debd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:38.705588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:38.706524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 242/486] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:38.707327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed3ac9-0993-4157-a7f7-277d552debd1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:38.809554 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed3ac9-0993-4157-a7f7-277d552debd1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:38.857770 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:38.858455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:38.858738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:38.859210 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:38.859769 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:38.860078 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:38.861508 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:38.874709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e114e7-6c14-49f0-9c3a-b11eb3cd5bb2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:38.878270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 243/487] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:38.883821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:38.910798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:38.911802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:38.912309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:38.912782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:38.913761 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:38.914268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:38.915828 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:38.924505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 245/488] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:38.930128 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.040867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce6f77-44ab-4d07-8a68-232f4f0241ad None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.070360 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a8333-9843-49b6-8e6c-aefc218da2d3 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.075298 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678eb3-fb6d-4d72-8a4c-5b90deab849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.075928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678eb3-fb6d-4d72-8a4c-5b90deab849f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.076208 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678eb3-fb6d-4d72-8a4c-5b90deab849f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.076478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678eb3-fb6d-4d72-8a4c-5b90deab849f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.087223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9678eb3-fb6d-4d72-8a4c-5b90deab849f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.088672 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9678eb3-fb6d-4d72-8a4c-5b90deab849f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f9678eb3-fb6d-4d72-8a4c-5b90deab849f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:39.107084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:39.107985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:39.108819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:39.108819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:39.108819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:39.108819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:39.108819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 244/489] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:39.109833 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778273e6-92b1-48ad-aa58-417671f2b641 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.110112 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778273e6-92b1-48ad-aa58-417671f2b641 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.110418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778273e6-92b1-48ad-aa58-417671f2b641 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.113117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 245/490] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:39.114082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778273e6-92b1-48ad-aa58-417671f2b641 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.118657 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f089ec-0e63-4246-9a88-03bdda8686cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.119709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f089ec-0e63-4246-9a88-03bdda8686cf None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.120180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f089ec-0e63-4246-9a88-03bdda8686cf None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.120649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f089ec-0e63-4246-9a88-03bdda8686cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.133559 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f089ec-0e63-4246-9a88-03bdda8686cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.135286 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f089ec-0e63-4246-9a88-03bdda8686cf None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-c0f089ec-0e63-4246-9a88-03bdda8686cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:39.153670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:39.154560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:39.155403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:39.155403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:39.155403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:39.155403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:39.157576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 246/491] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:39.163352 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500214ab-934b-45b6-ae99-d8ad1762ddd9 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.163844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500214ab-934b-45b6-ae99-d8ad1762ddd9 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.164325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500214ab-934b-45b6-ae99-d8ad1762ddd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.181662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-500214ab-934b-45b6-ae99-d8ad1762ddd9 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:39.183161 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-500214ab-934b-45b6-ae99-d8ad1762ddd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:39.223754 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778273e6-92b1-48ad-aa58-417671f2b641 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.253168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 247/492] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:39.258131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500214ab-934b-45b6-ae99-d8ad1762ddd9 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.259854 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.260755 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.261236 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.262190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.263213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:39.263745 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.265389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.280736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 246/493] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:39.286103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.375567 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500214ab-934b-45b6-ae99-d8ad1762ddd9 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.409674 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd640f87-0d9d-4dc3-8263-ce0fa43ff4dc None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.416176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.416765 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.417065 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.417357 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.418146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:39.418452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.420096 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.437744 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 248/494] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:39.438648 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.461950 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.463058 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.463506 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.463942 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.471605 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.473278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:39.493160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:39.494062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:02:39.494964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:39.497326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 247/495] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:39.497961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.545802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d243-0ffa-452d-af10-f17a264197b3 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.582932 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf817e-b058-449c-98b1-edee519a7642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.583723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf817e-b058-449c-98b1-edee519a7642 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.583996 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf817e-b058-449c-98b1-edee519a7642 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.584324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf817e-b058-449c-98b1-edee519a7642 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.594993 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf817e-b058-449c-98b1-edee519a7642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:39.600598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78741445-f457-4b9d-aeb1-705c1f1d08bb None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-bbaf817e-b058-449c-98b1-edee519a7642 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:39.602803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:39.603707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:39.604524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 249/496] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:39.605253 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6216b0-387e-46d1-a2d1-0375cb13b6a3 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.606182 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6216b0-387e-46d1-a2d1-0375cb13b6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.606303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6216b0-387e-46d1-a2d1-0375cb13b6a3 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.611566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 250/497] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:39.612258 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6216b0-387e-46d1-a2d1-0375cb13b6a3 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.638860 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.639947 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.640300 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.640572 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.647600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.649058 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:39.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:39.664943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:39.665987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 248/498] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:39.666590 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.713442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6216b0-387e-46d1-a2d1-0375cb13b6a3 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.756611 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274766e-7a1d-4e5c-bf8b-ec113cfd8116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.757318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274766e-7a1d-4e5c-bf8b-ec113cfd8116 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274766e-7a1d-4e5c-bf8b-ec113cfd8116 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.757876 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274766e-7a1d-4e5c-bf8b-ec113cfd8116 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.758718 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274766e-7a1d-4e5c-bf8b-ec113cfd8116 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:39.759030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274766e-7a1d-4e5c-bf8b-ec113cfd8116 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.760465 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274766e-7a1d-4e5c-bf8b-ec113cfd8116 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.772061 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead29d5-6dc3-439d-8dcc-678b9f3f3243 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.796840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 251/499] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:39.798149 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02403fec-2320-45b2-b368-b8e352717fca None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.798438 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02403fec-2320-45b2-b368-b8e352717fca None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.798717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02403fec-2320-45b2-b368-b8e352717fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.808268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.809064 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.809355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.809623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.810368 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:39.810673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.812038 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.814199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-02403fec-2320-45b2-b368-b8e352717fca None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:39.815384 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-02403fec-2320-45b2-b368-b8e352717fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:39.819693 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 249/500] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:39.820452 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.879047 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 252/501] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:39.880961 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c554-e6b8-44b0-bd37-bc53d6c47440 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.881282 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c554-e6b8-44b0-bd37-bc53d6c47440 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.881555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300c554-e6b8-44b0-bd37-bc53d6c47440 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.882793 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 253/502] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:39.883468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300c554-e6b8-44b0-bd37-bc53d6c47440 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:39.921494 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a288c18-8a55-4d06-b9aa-9fddb54ba671 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.952494 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:39.953177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:39.953451 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:39.953719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:39.954543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:39.954845 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:39.956254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:39.978097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300c554-e6b8-44b0-bd37-bc53d6c47440 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:39.991626 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 250/503] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:39.992423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.012624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bd03b-1eb1-4cbb-bc40-99884e28d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.013393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd03b-1eb1-4cbb-bc40-99884e28d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.013675 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd03b-1eb1-4cbb-bc40-99884e28d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.013946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bd03b-1eb1-4cbb-bc40-99884e28d6c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.081549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bd03b-1eb1-4cbb-bc40-99884e28d6c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.083004 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bd03b-1eb1-4cbb-bc40-99884e28d6c7 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.085674 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223aeabe-650c-4293-b6ef-37ecb437a6bf None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.120288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6333e-f9c8-4fba-8b0f-0d4ecdca6a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.121619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6333e-f9c8-4fba-8b0f-0d4ecdca6a03 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.121899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6333e-f9c8-4fba-8b0f-0d4ecdca6a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.122171 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6333e-f9c8-4fba-8b0f-0d4ecdca6a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.133473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6333e-f9c8-4fba-8b0f-0d4ecdca6a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.134968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6333e-f9c8-4fba-8b0f-0d4ecdca6a03 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-85e6333e-f9c8-4fba-8b0f-0d4ecdca6a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:40.152269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:40.153154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:40.154016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:40.154016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:40.154016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:40.154016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:40.154016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 251/504] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:40.154649 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b312b3b-d1ba-4d2f-b04b-6c8aca1a23d6 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.154894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b312b3b-d1ba-4d2f-b04b-6c8aca1a23d6 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.155161 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b312b3b-d1ba-4d2f-b04b-6c8aca1a23d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.170971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-3b312b3b-d1ba-4d2f-b04b-6c8aca1a23d6 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:40.172207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-3b312b3b-d1ba-4d2f-b04b-6c8aca1a23d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:40.181971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 254/505] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:40.182789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bd03b-1eb1-4cbb-bc40-99884e28d6c7 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.237658 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 252/506] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:40.238492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b312b3b-d1ba-4d2f-b04b-6c8aca1a23d6 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.276838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bd03b-1eb1-4cbb-bc40-99884e28d6c7 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.308099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c94a6-c8ab-45c9-b0d2-010d149a9e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.309420 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c94a6-c8ab-45c9-b0d2-010d149a9e15 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.309698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c94a6-c8ab-45c9-b0d2-010d149a9e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.309969 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c94a6-c8ab-45c9-b0d2-010d149a9e15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.319531 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c94a6-c8ab-45c9-b0d2-010d149a9e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-d68c94a6-c8ab-45c9-b0d2-010d149a9e15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:40.327695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:40.328789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 255/507] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:40.329861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c94a6-c8ab-45c9-b0d2-010d149a9e15 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.331065 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b312b3b-d1ba-4d2f-b04b-6c8aca1a23d6 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.362386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884eb1db-5977-4603-b624-eb8eb04dd001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.363034 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eb1db-5977-4603-b624-eb8eb04dd001 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.363311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eb1db-5977-4603-b624-eb8eb04dd001 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.363586 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eb1db-5977-4603-b624-eb8eb04dd001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.426835 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c94a6-c8ab-45c9-b0d2-010d149a9e15 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.427476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884eb1db-5977-4603-b624-eb8eb04dd001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.428938 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884eb1db-5977-4603-b624-eb8eb04dd001 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.458390 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.459037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.459317 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.459597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.460321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:40.460628 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.462019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.473187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 256/508] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:40.473981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.524716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 253/509] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:40.525576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884eb1db-5977-4603-b624-eb8eb04dd001 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.567303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013d943-fad7-4f8f-8578-d6ba13fcb5bd None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.599469 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.600240 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.600517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.600791 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.610375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.611818 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.616287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884eb1db-5977-4603-b624-eb8eb04dd001 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:40.628920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:40.630274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:40.631120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:40.631120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:40.631120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:40.631120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:40.631120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 257/510] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:40.631120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.653630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392608d-8408-4e14-81da-de0025176764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.654410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392608d-8408-4e14-81da-de0025176764 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.654692 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392608d-8408-4e14-81da-de0025176764 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.654962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392608d-8408-4e14-81da-de0025176764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.664489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392608d-8408-4e14-81da-de0025176764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.665946 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392608d-8408-4e14-81da-de0025176764 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-5392608d-8408-4e14-81da-de0025176764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:40.682547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:40.683437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:40.684301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:40.684301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:40.684301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:40.684301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:40.684301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 254/511] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:40.684564 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392608d-8408-4e14-81da-de0025176764 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.727298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f495c69-a6d9-4836-8b39-3dd139ad6960 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.758503 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.759156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.759438 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.759712 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.760452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:40.760761 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.762700 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.770205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 258/512] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:40.770974 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.775726 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392608d-8408-4e14-81da-de0025176764 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.811386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.812168 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.812445 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.812710 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.819514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.820953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:40.835423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:40.836483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 255/513] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:40.837414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.871871 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af66d7-a26b-412f-9b7b-3a2896fd849f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.906458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.907237 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.907514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.907784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.917733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.919695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.934625 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff69208-4a85-4248-b2a9-74b92522fbc0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:40.935887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:40.936852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:40.937913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:40.937913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:40.937913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:40.937913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:40.941327 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 259/514] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:40.942138 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:40.967351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:40.968000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:40.968279 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:40.968549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:40.969356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:40.969658 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:40.971021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:40.982231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 256/515] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:40.983105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.035689 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc035cc-d44e-4fa6-a6e1-3063efb0fa76 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.067097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.067742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.068023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.068298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.069012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:41.069350 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.070721 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.074440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49062fac-e98b-4b32-8251-142f5612a57b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.081941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 260/516] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:41.082722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.106831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.107690 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.108058 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.108338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.118354 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.119825 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:41.137779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:41.138683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:41.139839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:41.139839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:41.139839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:41.139839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:41.139839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 257/517] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:41.140252 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.185671 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e4bde-302d-41c0-9a7a-c85d0d438e4b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.217286 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.217957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.218241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.218517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.219358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:41.219668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.221129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.231930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca38aea-291e-4ba6-8eb5-1111f3f4560a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.238133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 261/518] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:41.238905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.263867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.264527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.264800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.265100 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.265819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:41.266129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.267484 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.278902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 258/519] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:41.279651 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.335479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec256185-0fc7-4a86-a0a8-17577fd62960 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.366846 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.367615 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.367890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.368156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.370318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11653185-a107-4e3f-9eb2-5bf937deb4f4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.375115 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.376558 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:41.390518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:41.391538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:41.392450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 262/520] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:41.392912 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.403053 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374a6de-e5a6-4a56-9c56-d2584cd7ac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.403794 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374a6de-e5a6-4a56-9c56-d2584cd7ac97 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.404058 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374a6de-e5a6-4a56-9c56-d2584cd7ac97 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.404334 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374a6de-e5a6-4a56-9c56-d2584cd7ac97 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.405070 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374a6de-e5a6-4a56-9c56-d2584cd7ac97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.406478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374a6de-e5a6-4a56-9c56-d2584cd7ac97 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.424806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 259/521] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:41 2022] POST /identity/v3/users => generated 247 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:41.425609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374a6de-e5a6-4a56-9c56-d2584cd7ac97 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.497293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8a2bb-7ee7-4291-bc4d-2b07602e6f52 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.533530 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.534651 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.535159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.535685 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.535789 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374a6de-e5a6-4a56-9c56-d2584cd7ac97 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.537150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:41.537622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.539056 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.556388 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 263/522] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:41.557394 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.571037 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede191c3-3281-4370-8fab-970b3d7176f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.571681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede191c3-3281-4370-8fab-970b3d7176f2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.571963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede191c3-3281-4370-8fab-970b3d7176f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.572236 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede191c3-3281-4370-8fab-970b3d7176f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.642086 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede191c3-3281-4370-8fab-970b3d7176f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.643550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede191c3-3281-4370-8fab-970b3d7176f2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.666242 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3800845-8273-42ed-9cd1-3d3d27e66618 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.698832 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.700172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.700445 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.700710 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.707101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.708525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:41.722454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:41.723330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:41.724219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 264/523] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:41.725267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.748494 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 260/524] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:41.748494 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede191c3-3281-4370-8fab-970b3d7176f2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.824313 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f206690-47df-4a1c-9592-c38823756577 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.841628 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede191c3-3281-4370-8fab-970b3d7176f2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.856991 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.857675 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.857965 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.858236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.859031 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:41.859343 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.860778 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.874270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:41.875054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:41.875335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:41.875602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:41.881351 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 265/525] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:41.882452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.883032 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:41.884458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:41.899843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:41.900830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:41.901777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 261/526] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:41.902915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:41.985602 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25393a-295a-4aeb-8495-3495de6fcf5e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:41.997541 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd23bc-8dd2-49c8-b947-a1783120e0b1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.019122 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.019711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.019992 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.020267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.021119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:42.021433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.022863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.031500 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.032315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.032597 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.032878 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.040123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 266/527] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:42.040123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:42.043133 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.044645 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:42.061898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:42.062780 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:42.063642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:42.063642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:42.063642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:42.063642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:42.063642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 262/528] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:42.063916 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:42.147548 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbca1a-adc9-4064-9291-9062a7b986ba None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.164046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86025b-b755-4714-87a4-a46b5159bdae None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.181849 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.182755 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.183215 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.184273 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.185134 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=512c115af57c463eb1fbe636667306dd) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:42.185450 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.187021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.198959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.199648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.199925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.200193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.201093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:42.201594 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.201809 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 267/529] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/role_assignments?user.id=512c115af57c463eb1fbe636667306dd&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:42.202947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:42.203059 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.233270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 263/530] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:42.234305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:42.319202 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa16a3a-94b8-42e5-96dd-425f8137d631 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.356584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.357714 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.358245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.358717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.359824 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=489243fe9de94fc693a06aa6f3183e18) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:42.360029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7117bb-869b-47f8-87cb-c04ac40e7dfe None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.360618 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.362321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.369087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 268/531] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/role_assignments?user.id=489243fe9de94fc693a06aa6f3183e18&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:42.370222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:42.407135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.407822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.408106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.408384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.409177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:42.409489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.410910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.444260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 264/532] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:42.458592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:42.485504 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a551c3-e54a-4183-8975-83ebbf51fdd4 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.523754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f1a1c-bb2d-45f6-a4db-6a6b696b4f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.525291 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f1a1c-bb2d-45f6-a4db-6a6b696b4f4e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.525788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f1a1c-bb2d-45f6-a4db-6a6b696b4f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.526261 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f1a1c-bb2d-45f6-a4db-6a6b696b4f4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.602454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f1a1c-bb2d-45f6-a4db-6a6b696b4f4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.604361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f1a1c-bb2d-45f6-a4db-6a6b696b4f4e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.664651 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc5f22-cd5d-4252-b93c-d6f487dbfa72 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.707606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.708740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.709259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.709784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.716851 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.718577 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:42.734454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:42.735370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:42.736217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:42.736217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:42.736217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:42.736217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:42.736217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:42.736217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:42.736217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:42.736217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:42.738764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 265/533] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:42.744368 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:42.811725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 269/534] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:42.817480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f1a1c-bb2d-45f6-a4db-6a6b696b4f4e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:42.917096 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f1a1c-bb2d-45f6-a4db-6a6b696b4f4e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.951332 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:42.952440 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:42.952922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:42.953445 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:42.963800 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:42.966089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:42.971788 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d3a1-22d2-4e38-9080-5d53b8ff4013 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:42.986666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:42.987587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:42.988443 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:42.988443 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:42.988443 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:42.988443 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:42.990635 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 270/535] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:42.996756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:43.030028 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:43.030689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:43.030962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:43.031232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:43.032023 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:43.032326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:43.033800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:43.065448 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 266/536] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:43.082455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:43.137231 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5e51a-4aff-4e39-93fa-6d37060040fd None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:43.178978 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:43.179555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:43.179829 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:43.180098 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:43.180817 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:43.181172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:43.182553 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:43.204018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 271/537] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:43.210578 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:43.276711 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1264d1-9b59-4bdc-ad4b-c04645310a41 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:43.332910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e837-bfdb-48fc-abbf-2cbf2ae120c9 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:43.343067 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:43.343732 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:43.344010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:43.344280 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:43.345144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=796bb66e8c14434097fde50e6815c9cd) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:43.345455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:43.346881 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:43.361153 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 267/538] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:43 2022] GET /identity/v3/role_assignments?user.id=796bb66e8c14434097fde50e6815c9cd&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:43.378157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:43.379015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:43.379560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:43.379834 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:43.391394 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:43.392881 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:43.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:43.409342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:43.410209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 272/539] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:43.416475 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:43.539283 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e820e2-1a3b-41d0-a88a-710a06fde353 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:43.575370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c710f42-7a11-4970-a5d7-3df5762e3741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:43.576038 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710f42-7a11-4970-a5d7-3df5762e3741 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:43.576317 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710f42-7a11-4970-a5d7-3df5762e3741 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:43.576587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710f42-7a11-4970-a5d7-3df5762e3741 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:43.577426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c710f42-7a11-4970-a5d7-3df5762e3741 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:43.577736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c710f42-7a11-4970-a5d7-3df5762e3741 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:43.579179 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c710f42-7a11-4970-a5d7-3df5762e3741 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:43.592210 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 268/540] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:43.598021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:43.703015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07de8c3-cea9-442d-a703-ad41ab964cc8 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:43.738453 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf00fd-db0e-43f4-87f6-c25fca3e6f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:43.739104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf00fd-db0e-43f4-87f6-c25fca3e6f95 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:43.739384 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf00fd-db0e-43f4-87f6-c25fca3e6f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:43.739657 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf00fd-db0e-43f4-87f6-c25fca3e6f95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:43.740489 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf00fd-db0e-43f4-87f6-c25fca3e6f95 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=adf51a2d99154eccbc74f0c63fc18bf9) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:43.740798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf00fd-db0e-43f4-87f6-c25fca3e6f95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:43.742259 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf00fd-db0e-43f4-87f6-c25fca3e6f95 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:43.748143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 273/541] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:43 2022] GET /identity/v3/role_assignments?user.id=adf51a2d99154eccbc74f0c63fc18bf9&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:43.945335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf38126-ea10-4ef2-903b-379330e8fa9f None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:43.945900 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf38126-ea10-4ef2-903b-379330e8fa9f None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:43.946157 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf38126-ea10-4ef2-903b-379330e8fa9f None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:43.948057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 269/542] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:43.953825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa3aa4-b3be-41d6-97de-682bcaf60b1e None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:43.954205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa3aa4-b3be-41d6-97de-682bcaf60b1e None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:43.954576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa3aa4-b3be-41d6-97de-682bcaf60b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:43.972684 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-68fa3aa4-b3be-41d6-97de-682bcaf60b1e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:43.973934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-68fa3aa4-b3be-41d6-97de-682bcaf60b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:44.045934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 274/543] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:44.074713 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf38126-ea10-4ef2-903b-379330e8fa9f None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:44.182286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf38126-ea10-4ef2-903b-379330e8fa9f None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:44.218272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795d6d0-a5e0-4026-acb6-ad7175eaabbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:44.218986 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795d6d0-a5e0-4026-acb6-ad7175eaabbb None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:44.219262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795d6d0-a5e0-4026-acb6-ad7175eaabbb None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:44.219534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795d6d0-a5e0-4026-acb6-ad7175eaabbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:44.230224 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795d6d0-a5e0-4026-acb6-ad7175eaabbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-c795d6d0-a5e0-4026-acb6-ad7175eaabbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:44.238084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:44.238957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:44.239775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 270/544] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:44.245443 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa3aa4-b3be-41d6-97de-682bcaf60b1e None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:44.345557 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa3aa4-b3be-41d6-97de-682bcaf60b1e None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:44.380588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:44.381279 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:44.381616 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:44.381890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:44.382693 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:44.382999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:44.384368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:44.392900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 275/545] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:44.398487 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795d6d0-a5e0-4026-acb6-ad7175eaabbb None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:44.505201 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795d6d0-a5e0-4026-acb6-ad7175eaabbb None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:44.539327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eeac61-eb92-41a7-a889-9f84053d574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:44.540142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeac61-eb92-41a7-a889-9f84053d574d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:44.540422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeac61-eb92-41a7-a889-9f84053d574d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:44.540703 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeac61-eb92-41a7-a889-9f84053d574d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:44.553435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeac61-eb92-41a7-a889-9f84053d574d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:44.554943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeac61-eb92-41a7-a889-9f84053d574d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-74eeac61-eb92-41a7-a889-9f84053d574d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:44.575022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:44.577075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:44.579092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:44.579092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:44.579092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:44.579092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:44.579092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 271/546] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:44.582070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:44.684686 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ced1cb-ca6a-4864-b80a-1331cfd9070d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:44.717873 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:44.718535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:44.718812 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:44.719085 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:44.719809 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:44.720114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:44.721534 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:44.734027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 276/547] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:44.738611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bdd14-7aee-4e9c-9bd2-dd6bdf618971 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:44.738896 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bdd14-7aee-4e9c-9bd2-dd6bdf618971 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:44.739174 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bdd14-7aee-4e9c-9bd2-dd6bdf618971 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:44.740634 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 272/548] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:44.742934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:44.746317 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cb4bd-f681-444f-a818-06313cd940fa None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:44.746604 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cb4bd-f681-444f-a818-06313cd940fa None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:44.746881 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cb4bd-f681-444f-a818-06313cd940fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:44.774286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-af5cb4bd-f681-444f-a818-06313cd940fa None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:44.775455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-af5cb4bd-f681-444f-a818-06313cd940fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:44.856251 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b7a7f-a55c-4870-881e-1f5e533ae303 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:44.899572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:44.900370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:44.900647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:44.900915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:44.909666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:44.911402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:44.929150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:44.930071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:44.930948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 277/549] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:44.936071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:44.954391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 273/550] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:44.994550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cb4bd-f681-444f-a818-06313cd940fa None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:45.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfc2eb-0622-4654-bf4c-142cecb3e765 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:45.088819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:45.090226 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.090510 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.090786 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.091632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:45.092099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:45.093597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:45.114851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 278/551] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:02:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:45.121535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:45.220370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bdaa6-e393-4c75-81a7-c84860c4d1bb None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:45.253267 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cb4bd-f681-444f-a818-06313cd940fa None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:45.255135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab72dd9-9357-4021-a6ba-60a0079ea890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:45.255795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72dd9-9357-4021-a6ba-60a0079ea890 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.256073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72dd9-9357-4021-a6ba-60a0079ea890 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.256363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72dd9-9357-4021-a6ba-60a0079ea890 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.294163 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfcd8f-5fe5-4261-9273-3333e2409c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:45.295020 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfcd8f-5fe5-4261-9273-3333e2409c5e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.295299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfcd8f-5fe5-4261-9273-3333e2409c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.295625 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfcd8f-5fe5-4261-9273-3333e2409c5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.305932 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfcd8f-5fe5-4261-9273-3333e2409c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-77bfcd8f-5fe5-4261-9273-3333e2409c5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:45.313532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:45.314438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:45.315248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 274/552] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:45.316222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece27bfc-280a-48b1-851b-becb579e7684 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.316507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece27bfc-280a-48b1-851b-becb579e7684 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.316785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece27bfc-280a-48b1-851b-becb579e7684 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.322536 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 275/553] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:45.323276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece27bfc-280a-48b1-851b-becb579e7684 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:45.337207 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab72dd9-9357-4021-a6ba-60a0079ea890 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:45.340058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab72dd9-9357-4021-a6ba-60a0079ea890 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:45.428327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece27bfc-280a-48b1-851b-becb579e7684 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:45.461925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:45.462719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.463073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.463418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.464247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:45.465193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:45.466725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:45.473235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 279/554] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:02:45.474561 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64743a6-b85d-4598-9b92-20a3348af893 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.474853 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64743a6-b85d-4598-9b92-20a3348af893 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.475130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64743a6-b85d-4598-9b92-20a3348af893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.478363 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 276/555] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:45.479466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:45.494614 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-b64743a6-b85d-4598-9b92-20a3348af893 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:45.496631 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-b64743a6-b85d-4598-9b92-20a3348af893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:45.573389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 280/556] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:45.574795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64743a6-b85d-4598-9b92-20a3348af893 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:45.600952 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5f6e0-d5d5-4ccd-b898-8138820f918f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:45.646266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:45.647393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.647974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.648245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.661584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:45.663495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:45.686401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:45.687266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:45.688103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:45.688103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:45.688103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:45.688103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:45.688321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64743a6-b85d-4598-9b92-20a3348af893 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:45.688717 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 277/557] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:45.688794 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:45.722650 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:45.723502 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.723784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.724108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.734776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:45.736228 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:45.755708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:45.757749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:45.759856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:45.759856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:45.759856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:45.759856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:45.759856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 281/558] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:45.759856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:45.811972 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ceb46-a975-4f8e-8141-dc955d203500 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:45.853705 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605ac99-5b86-4cc1-8de3-20c954903534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:45.854575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605ac99-5b86-4cc1-8de3-20c954903534 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.855671 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605ac99-5b86-4cc1-8de3-20c954903534 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.855954 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605ac99-5b86-4cc1-8de3-20c954903534 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.863290 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e62be-e358-470b-bd96-b7e2a371e56b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:45.868141 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605ac99-5b86-4cc1-8de3-20c954903534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f605ac99-5b86-4cc1-8de3-20c954903534 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:45.878275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:45.880259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 278/559] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:45.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605ac99-5b86-4cc1-8de3-20c954903534 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:45.904364 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:45.905149 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:45.905502 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:45.905853 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:45.906651 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:45.907048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:45.908478 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:45.920894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 282/560] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:45.921889 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.012574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605ac99-5b86-4cc1-8de3-20c954903534 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.035429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79b75d-866d-4a9d-be07-acf7b9a27391 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.053883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.054548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.054824 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.055095 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.055814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:46.056116 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.057923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.071367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.072046 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.072361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.072696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.073484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:46.073817 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.075834 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.077128 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 279/561] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:46.078837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.089204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 283/562] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:46.090146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.191708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912c9ad-46ee-4be7-adb0-3367b334a473 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.204746 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18155c7c-1599-4bc2-baba-03c30a8e39d5 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.226105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.226894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.227172 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.227446 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.233981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.235414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.240287 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c2b6c-e552-4a3e-988b-b27ec2f5fdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.241556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2b6c-e552-4a3e-988b-b27ec2f5fdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.242030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2b6c-e552-4a3e-988b-b27ec2f5fdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.242475 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c2b6c-e552-4a3e-988b-b27ec2f5fdc5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:46.249427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:46.250799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:46.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:46.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:46.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:46.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:46.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:46.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:46.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:46.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:46.252240 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2b6c-e552-4a3e-988b-b27ec2f5fdc5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.253126 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c2b6c-e552-4a3e-988b-b27ec2f5fdc5 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.255648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 280/563] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:46.256342 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-ba5c2b6c-e552-4a3e-988b-b27ec2f5fdc5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:46.268171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:46.269135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:46.270055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 284/564] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:46.272180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e271f-0563-4b40-8e22-71c9412b8f2d None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.272180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e271f-0563-4b40-8e22-71c9412b8f2d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.272180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e271f-0563-4b40-8e22-71c9412b8f2d None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.277535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 285/565] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:46.277535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e271f-0563-4b40-8e22-71c9412b8f2d None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.363263 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd79f7-e81d-403f-87ba-df53da616b9a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.385560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e271f-0563-4b40-8e22-71c9412b8f2d None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.399106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.399893 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.400166 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.400436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.412705 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.414192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.418803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd3c23-eb3c-4f16-a1b7-03aff932c317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.419456 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3c23-eb3c-4f16-a1b7-03aff932c317 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.419730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3c23-eb3c-4f16-a1b7-03aff932c317 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.419997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd3c23-eb3c-4f16-a1b7-03aff932c317 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.420781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd3c23-eb3c-4f16-a1b7-03aff932c317 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:46.421109 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd3c23-eb3c-4f16-a1b7-03aff932c317 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.422581 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd3c23-eb3c-4f16-a1b7-03aff932c317 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:46.431212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:46.432223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:46.434262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:46.434262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:46.434262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:46.434262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:46.437277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 281/566] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:46.438168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 286/567] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:46.438498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.439632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43246c-e182-4376-a310-79b3cc610515 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.439977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43246c-e182-4376-a310-79b3cc610515 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.440362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43246c-e182-4376-a310-79b3cc610515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.456734 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-8b43246c-e182-4376-a310-79b3cc610515 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:46.457946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-8b43246c-e182-4376-a310-79b3cc610515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:46.530362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 287/568] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:46.531241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43246c-e182-4376-a310-79b3cc610515 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.541735 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212024f6-984e-42c1-9bca-05c71b5c134e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.574505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.575167 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.575447 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.575723 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.576552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:46.576865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.578320 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.595063 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 282/569] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:46.595827 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.625685 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43246c-e182-4376-a310-79b3cc610515 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.657987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059b44a-c64e-4c8c-8d58-69eddaff6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.658638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059b44a-c64e-4c8c-8d58-69eddaff6db4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.658925 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059b44a-c64e-4c8c-8d58-69eddaff6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.659198 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059b44a-c64e-4c8c-8d58-69eddaff6db4 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.659913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059b44a-c64e-4c8c-8d58-69eddaff6db4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:46.660223 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059b44a-c64e-4c8c-8d58-69eddaff6db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.661604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059b44a-c64e-4c8c-8d58-69eddaff6db4 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.672802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 288/570] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:46.674027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fe5b4-1ec3-4fe4-b584-0a6ba3d6ed79 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.674312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fe5b4-1ec3-4fe4-b584-0a6ba3d6ed79 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.674588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fe5b4-1ec3-4fe4-b584-0a6ba3d6ed79 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.675930 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 289/571] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:46.677843 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fe5b4-1ec3-4fe4-b584-0a6ba3d6ed79 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.688714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e51ce-4353-4aae-9470-73000c57cfd7 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.721892 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.722620 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.722903 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.723998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=14a8942a978b4d92be89376cd31a1240) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:46.724305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.725771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.731827 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 283/572] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/role_assignments?user.id=14a8942a978b4d92be89376cd31a1240&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:46.733177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.801488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fe5b4-1ec3-4fe4-b584-0a6ba3d6ed79 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.841535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5de63e-463f-4b44-8dee-bae88997187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.842349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5de63e-463f-4b44-8dee-bae88997187e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.842631 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5de63e-463f-4b44-8dee-bae88997187e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.842904 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5de63e-463f-4b44-8dee-bae88997187e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.854017 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5de63e-463f-4b44-8dee-bae88997187e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-0a5de63e-463f-4b44-8dee-bae88997187e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:46.862072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:46.863897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:46.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:46.865745 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7752d73-afaf-4130-89f1-756e21d154e5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:46.865821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 290/573] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:46.865821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5de63e-463f-4b44-8dee-bae88997187e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:46.907162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cebb63-5d38-4e8e-a436-79df0fa29f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:46.907961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cebb63-5d38-4e8e-a436-79df0fa29f68 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.908283 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cebb63-5d38-4e8e-a436-79df0fa29f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.908499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cebb63-5d38-4e8e-a436-79df0fa29f68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.921612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cebb63-5d38-4e8e-a436-79df0fa29f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:46.923059 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cebb63-5d38-4e8e-a436-79df0fa29f68 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-61cebb63-5d38-4e8e-a436-79df0fa29f68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:46.941575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:46.942678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:46.943831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:46.943831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:46.943831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:46.943831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:46.949909 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 284/574] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:46.949909 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0910c-df80-4559-8177-5919a5360e32 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:46.949909 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0910c-df80-4559-8177-5919a5360e32 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:46.949909 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0910c-df80-4559-8177-5919a5360e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:46.966796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-73e0910c-df80-4559-8177-5919a5360e32 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:46.968377 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-73e0910c-df80-4559-8177-5919a5360e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:46.984104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5de63e-463f-4b44-8dee-bae88997187e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.018545 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.019473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.019853 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.020222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.028444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.030023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.046553 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 285/575] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:47.046553 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0910c-df80-4559-8177-5919a5360e32 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:47.047143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:47.048197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 291/576] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:47.049403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.157268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2f4d1-14d3-43e6-a5ac-fb7de3af0e00 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.169575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0910c-df80-4559-8177-5919a5360e32 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.194687 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.195362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.195675 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.195973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.196951 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:47.197171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.198549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.208735 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.209449 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.209727 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.209997 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.210744 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:47.215971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.216098 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 292/577] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:47.216098 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.216256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.221510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 286/578] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:47.226293 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.318095 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b988e0-dac3-4fff-b38d-18bf86d9e477 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.325243 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1aa6b-5c7c-485e-83f7-dd4b5a65ab0f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.352718 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.353415 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.353694 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.353967 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.354797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:47.355633 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.357064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.358431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117503-8ae4-43b0-a337-765663c2586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.359208 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117503-8ae4-43b0-a337-765663c2586f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.359485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117503-8ae4-43b0-a337-765663c2586f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.359751 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf117503-8ae4-43b0-a337-765663c2586f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.369735 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf117503-8ae4-43b0-a337-765663c2586f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:47.378947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 293/579] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:47.378947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-cf117503-8ae4-43b0-a337-765663c2586f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:47.379143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:47.380276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 287/580] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:47.381790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117503-8ae4-43b0-a337-765663c2586f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.479898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf117503-8ae4-43b0-a337-765663c2586f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.481727 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9dad51-b839-4e70-94d8-5b01bcb04e27 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.513489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.514280 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.514559 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.514829 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.516403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.516682 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.516955 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.523613 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.525068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.525173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.526535 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:47.539107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:47.540016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 294/581] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:47.546402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:47.546981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:47.547763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:47.548636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:47.548636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:47.548636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:47.548636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:47.548636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 288/582] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:47.548636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.644980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956438d-2c06-4df3-85ea-4e458e74265e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.652951 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ccd0a-e16a-4f7f-b4dc-9d13a64fed27 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.677089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.677756 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.678036 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.678313 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.679051 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:47.679359 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.680725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.686233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.687018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.687277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.687552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.689225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 289/583] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:47.689788 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.703268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.704743 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:47.725914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:47.727285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:47.728398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:47.728398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:47.728398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:47.728398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:47.728398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 295/584] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:47.728398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.794357 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b822c-bda5-425e-9703-27e0beaae891 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.825390 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bb842-6cf9-4e04-85df-ec34370b16fe None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.829923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.830581 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.830859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.831131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.831918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:47.832229 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.833676 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.849838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 290/585] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:47.849838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.863076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.864109 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.864551 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.864818 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.865613 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:47.865919 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:47.867353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:47.881399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 296/586] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:47.882461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:47.954255 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040f3db-d43b-42e2-bbe6-e83596d009cc None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.978841 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7235675-645d-4a5a-9d4c-856c25d23b12 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:47.987247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:47.988030 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:47.988308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:47.988578 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:47.999138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.000577 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.013626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.014279 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.014559 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.014832 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.015546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:48.015852 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.017231 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:48.018416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:48.019351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:48.020231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:48.020231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:48.020231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:48.020231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:48.020231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 291/587] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:48.020514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.035777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 297/588] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:48.036589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.127372 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3affba-b4bf-4017-9b7e-d61c74463d6b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.132371 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f80f2-f62d-491c-ae98-bc38e8e95fe4 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.164972 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.165177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488e0c4-3a2d-41cf-bd69-96992f2e12fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.165796 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.165908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488e0c4-3a2d-41cf-bd69-96992f2e12fd None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.166072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.166175 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488e0c4-3a2d-41cf-bd69-96992f2e12fd None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.166347 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.166456 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488e0c4-3a2d-41cf-bd69-96992f2e12fd None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/512c115af57c463eb1fbe636667306dd/roles/48c24d23f41249abbb5224c3fa457f9e` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.173774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.175206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.180259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488e0c4-3a2d-41cf-bd69-96992f2e12fd None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=512c115af57c463eb1fbe636667306dd, role_id=48c24d23f41249abbb5224c3fa457f9e)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.181708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488e0c4-3a2d-41cf-bd69-96992f2e12fd None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:48.189225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:48.190209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:02:48.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:48.195263 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 298/589] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:48.195829 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.209378 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 292/590] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:48 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/512c115af57c463eb1fbe636667306dd/roles/48c24d23f41249abbb5224c3fa457f9e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:48.209378 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488e0c4-3a2d-41cf-bd69-96992f2e12fd None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.329406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488e0c4-3a2d-41cf-bd69-96992f2e12fd None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.336091 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aafe93-7498-4e3d-acf0-f7b8f1f6acfc None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.370912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.371717 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.371994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.372264 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.373325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.374060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.374411 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.374808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.375564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:48.376021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.377602 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.380032 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.381505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.391107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 299/591] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:48.391829 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:48.401054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:48.402630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 293/592] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:48.404624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.499612 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf57a35-c3a3-4fc4-996a-309d6a4ed5f3 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.511976 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2848dd5-15e7-4b0f-8add-d39b398cf1d0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.534112 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.534785 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.535065 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.535391 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.536235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:48.536542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.537999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.546198 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.546990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.547270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.547545 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.555411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.556928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.561429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 300/593] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:48.562323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:48.572441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:48.573551 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:02:48.575155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:48.581078 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 294/594] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:48.581078 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.662666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3440cb3-d57a-47a9-b0b0-02856645e344 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.680618 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63948d-9f69-42c1-abab-c83372c5973e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.697126 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.697802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.698058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.698333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.699208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:48.699445 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.700938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.717130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 301/595] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:48.717130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.721269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.722274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.722716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.723135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.735137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.736775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:48.755033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:48.756144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:48.757176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:48.757176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:48.757176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:48.757176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:48.759092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 295/596] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:48.759994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.832979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe2613-2179-474c-860a-15eb021cd61d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.870615 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.871795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.872321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.872452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.873272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:48.873580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.875020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.889760 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f28df-7508-4994-b53e-cb363d83f43d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:48.907473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 302/597] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:48.909083 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:48.930218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7c253-9503-44c4-b84f-bdb240d361e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:48.931337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c253-9503-44c4-b84f-bdb240d361e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:48.931777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c253-9503-44c4-b84f-bdb240d361e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:48.932250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c253-9503-44c4-b84f-bdb240d361e6 None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/489243fe9de94fc693a06aa6f3183e18/roles/48c24d23f41249abbb5224c3fa457f9e` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:48.951466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7c253-9503-44c4-b84f-bdb240d361e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=489243fe9de94fc693a06aa6f3183e18, role_id=48c24d23f41249abbb5224c3fa457f9e)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:48.953262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7c253-9503-44c4-b84f-bdb240d361e6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:48.969903 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 296/598] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:48 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/489243fe9de94fc693a06aa6f3183e18/roles/48c24d23f41249abbb5224c3fa457f9e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:48.978167 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7c253-9503-44c4-b84f-bdb240d361e6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.012533 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84d2a7-062a-4006-901b-94307c556162 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.049073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f759d1f-4628-478c-8566-2bbdc1931a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.053989 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f759d1f-4628-478c-8566-2bbdc1931a09 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.054272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f759d1f-4628-478c-8566-2bbdc1931a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.054550 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f759d1f-4628-478c-8566-2bbdc1931a09 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.055582 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f759d1f-4628-478c-8566-2bbdc1931a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:49.055582 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f759d1f-4628-478c-8566-2bbdc1931a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:49.056999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f759d1f-4628-478c-8566-2bbdc1931a09 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:49.069667 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 303/599] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:49.071072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca98d26-dc74-4241-aa69-0ee2c24fa9f2 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.071354 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca98d26-dc74-4241-aa69-0ee2c24fa9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.071626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca98d26-dc74-4241-aa69-0ee2c24fa9f2 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.077181 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 304/600] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:49.078052 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca98d26-dc74-4241-aa69-0ee2c24fa9f2 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.112762 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7c253-9503-44c4-b84f-bdb240d361e6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.161985 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf51ae-fcdb-441d-aa61-b3f64f0bbb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.164592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf51ae-fcdb-441d-aa61-b3f64f0bbb86 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.165094 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf51ae-fcdb-441d-aa61-b3f64f0bbb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.165552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf51ae-fcdb-441d-aa61-b3f64f0bbb86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.179530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf51ae-fcdb-441d-aa61-b3f64f0bbb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:49.181940 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf51ae-fcdb-441d-aa61-b3f64f0bbb86 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:49.201701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca98d26-dc74-4241-aa69-0ee2c24fa9f2 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-92cf51ae-fcdb-441d-aa61-b3f64f0bbb86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:49.207507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:49.208462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:49.209422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:49.209422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:49.209422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:49.209422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:49.213123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 297/601] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:49.215644 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052d0b-470d-419b-a5b3-bd69b957b70b None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.216292 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052d0b-470d-419b-a5b3-bd69b957b70b None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.216786 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052d0b-470d-419b-a5b3-bd69b957b70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.235684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-84052d0b-470d-419b-a5b3-bd69b957b70b None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:49.236890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-84052d0b-470d-419b-a5b3-bd69b957b70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:49.268032 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.269787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.270634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.271357 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.285513 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:49.288477 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:49.311436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 298/602] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:49.312278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052d0b-470d-419b-a5b3-bd69b957b70b None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:49.317520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:49.318400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:49.319270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 305/603] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:49.324847 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.424082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052d0b-470d-419b-a5b3-bd69b957b70b None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.453545 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f8b5-c25c-46cf-af78-ec6462c461db None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.458557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.459458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.459834 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.460209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.461117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:49.461523 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:49.462987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:49.475687 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 299/604] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:49.481168 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.493006 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.493694 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.493969 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.494547 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.495668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:49.496046 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:49.498197 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:49.518270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 306/605] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:49.519245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.607408 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0e4e6-4d85-493b-aeb8-85214b0a83f1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.621629 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dab929-4c00-47af-a27b-48efe44f56fe None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.653134 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81aeb27-db66-437d-9351-c36083d218b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.654229 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aeb27-db66-437d-9351-c36083d218b1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.654760 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aeb27-db66-437d-9351-c36083d218b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.655035 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aeb27-db66-437d-9351-c36083d218b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.658517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.659373 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.659736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.660068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.665493 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81aeb27-db66-437d-9351-c36083d218b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:49.667142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:49.669245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-d81aeb27-db66-437d-9351-c36083d218b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:49.674367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:49.676137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:49.677367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:49.685079 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 300/606] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:49.685079 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81aeb27-db66-437d-9351-c36083d218b1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:49.689717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:49.691234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:49.692915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:49.692915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:49.692915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:49.692915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:49.692915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:49.692915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:49.692915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:49.692915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:49.697162 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 307/607] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:49.698056 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.800314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e148b-81b1-4743-9fa0-baab3a41a000 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.803397 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81aeb27-db66-437d-9351-c36083d218b1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:49.834580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.835361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.835706 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.836043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.836859 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:49.837254 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:49.838789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:49.842400 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1adb0-01c4-4081-ba2b-874b59415df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:49.843284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1adb0-01c4-4081-ba2b-874b59415df6 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:49.843556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1adb0-01c4-4081-ba2b-874b59415df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:49.843820 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1adb0-01c4-4081-ba2b-874b59415df6 None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/796bb66e8c14434097fde50e6815c9cd/roles/48c24d23f41249abbb5224c3fa457f9e` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:49.852789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 308/608] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:49.853717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.860866 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1adb0-01c4-4081-ba2b-874b59415df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=796bb66e8c14434097fde50e6815c9cd, role_id=48c24d23f41249abbb5224c3fa457f9e)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:49.862341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1adb0-01c4-4081-ba2b-874b59415df6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:49.889268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 301/609] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:49 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/796bb66e8c14434097fde50e6815c9cd/roles/48c24d23f41249abbb5224c3fa457f9e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:49.890272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1adb0-01c4-4081-ba2b-874b59415df6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:49.977024 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcd06f-c394-4533-98e3-99c8c9183867 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:50.018308 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:50.019274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:50.019551 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:50.019824 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:50.020679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:50.020989 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:50.022930 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:50.037393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 309/610] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:50.047482 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:50.147198 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1adb0-01c4-4081-ba2b-874b59415df6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:50.166622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74440375-78f4-4539-97b3-02c63aefc7aa None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:50.209341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:50.210149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:50.210427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:50.210710 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:50.217841 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0abe7-e5ac-4379-a28b-4868080f147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:50.218737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0abe7-e5ac-4379-a28b-4868080f147f None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:50.219014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0abe7-e5ac-4379-a28b-4868080f147f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:50.219285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0abe7-e5ac-4379-a28b-4868080f147f None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/adf51a2d99154eccbc74f0c63fc18bf9/roles/48c24d23f41249abbb5224c3fa457f9e` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:50.225713 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:50.227171 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:50.238454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0abe7-e5ac-4379-a28b-4868080f147f None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=adf51a2d99154eccbc74f0c63fc18bf9, role_id=48c24d23f41249abbb5224c3fa457f9e)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:50.239889 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0abe7-e5ac-4379-a28b-4868080f147f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:50.248281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:50.249294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:50.250200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:50.250200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:50.250200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:50.250200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:50.250200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 302/611] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:50.256113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:50.256994 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 310/612] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:50 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/adf51a2d99154eccbc74f0c63fc18bf9/roles/48c24d23f41249abbb5224c3fa457f9e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:02:50.390709 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989b70f-cdf1-4593-a6e6-5b461e1b9fb3 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:50.439804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a2534-837b-4425-99a9-465ca332693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:50.440530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a2534-837b-4425-99a9-465ca332693c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:50.440819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a2534-837b-4425-99a9-465ca332693c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:50.441113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a2534-837b-4425-99a9-465ca332693c None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:50.441905 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a2534-837b-4425-99a9-465ca332693c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:50.442213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a2534-837b-4425-99a9-465ca332693c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:50.443662 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a2534-837b-4425-99a9-465ca332693c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:50.456734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 303/613] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:50.462350 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0abe7-e5ac-4379-a28b-4868080f147f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:50.600599 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0abe7-e5ac-4379-a28b-4868080f147f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:50.637942 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:50.638791 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:50.639071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:50.639344 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:50.646646 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:50.648018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:50.663035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:50.664268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:02:50.665429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:50.666037 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1850d-a5c0-4866-9600-f7b9bd0f4928 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:50.666037 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1850d-a5c0-4866-9600-f7b9bd0f4928 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:50.666037 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1850d-a5c0-4866-9600-f7b9bd0f4928 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:50.666271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 311/614] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:50.666351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 304/615] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:50.671528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:50.675800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcc516-328f-48fc-aa9b-501a678dc1f4 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:50.676077 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcc516-328f-48fc-aa9b-501a678dc1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:50.676349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcc516-328f-48fc-aa9b-501a678dc1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:50.707613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-79fcc516-328f-48fc-aa9b-501a678dc1f4 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:50.708802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-79fcc516-328f-48fc-aa9b-501a678dc1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:50.797947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5b21-9752-4c3d-8e7e-899698a17ab6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:50.841118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:50.842600 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:50.843104 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:50.843570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:50.844618 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:50.846978 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:50.848630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:50.867328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 305/616] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:02:50.876626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 312/617] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:02:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:50.882140 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcc516-328f-48fc-aa9b-501a678dc1f4 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:50.909719 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:50.993705 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcc516-328f-48fc-aa9b-501a678dc1f4 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.028188 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.028993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.029335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.029571 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.036069 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1253bd6-7c49-4246-9458-02c989dbc375 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.039790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:51.041249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:51.059763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:51.060721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:51.061647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:51.061647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:51.061647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:51.061647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:51.065146 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 306/618] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:51.066686 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:51.082757 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b87cb1-5d10-43fc-85c6-b9fa0490524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.083893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87cb1-5d10-43fc-85c6-b9fa0490524a None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.084378 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87cb1-5d10-43fc-85c6-b9fa0490524a None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.084848 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87cb1-5d10-43fc-85c6-b9fa0490524a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.106658 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b87cb1-5d10-43fc-85c6-b9fa0490524a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-39b87cb1-5d10-43fc-85c6-b9fa0490524a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:51.123153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:51.124137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:51.124990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:51.127977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 313/619] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:51.143887 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b87cb1-5d10-43fc-85c6-b9fa0490524a None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:51.179713 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ee7c2-2245-44e5-a2d9-bee7a74a0f0e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.219035 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.219810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.219978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.220235 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.220953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:51.221285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:51.222668 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:51.233925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 307/620] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:51.239350 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:51.337762 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b87cb1-5d10-43fc-85c6-b9fa0490524a None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.339972 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960d277-ce04-43fa-b01b-648b0cde8b18 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.379323 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.380123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.380397 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.380666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.393401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:51.394836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:51.401360 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.403014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.403474 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.403917 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.405155 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:51.405897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:51.408509 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:51.421304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:51.422220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:51.423111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:51.423111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:51.423111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:51.423111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:51.423111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:51.423111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:51.423111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:51.423111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:51.423788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 314/621] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:51.425937 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:51.430367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 308/622] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:51.442087 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:51.586891 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0b3ba-dc1e-447f-bb1a-a05ab503ab06 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.601023 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4feaacb-5299-4470-8e38-655792de2267 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.624878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4faa7-c0b3-4540-b179-178da39abd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.625710 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4faa7-c0b3-4540-b179-178da39abd62 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.626129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4faa7-c0b3-4540-b179-178da39abd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.626260 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4faa7-c0b3-4540-b179-178da39abd62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.637275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.637769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4faa7-c0b3-4540-b179-178da39abd62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:51.638269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.638688 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.639125 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.639237 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4faa7-c0b3-4540-b179-178da39abd62 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:51.640342 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:51.640800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:51.642365 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-2df4faa7-c0b3-4540-b179-178da39abd62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:51.659097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:51.660178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:51.661216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:51.661216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:51.661216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:51.661216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:51.661216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 315/624] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:51.661216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b9fb6-6e87-4171-bc6b-1ad37dcef666 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.661216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b9fb6-6e87-4171-bc6b-1ad37dcef666 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.661216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b9fb6-6e87-4171-bc6b-1ad37dcef666 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.661813 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 309/624] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:51.664182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:51.664825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 316/625] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:51.665545 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b9fb6-6e87-4171-bc6b-1ad37dcef666 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:51.766485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b9fb6-6e87-4171-bc6b-1ad37dcef666 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.767215 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529ddea-5a83-4cb5-9ea1-73b9fc052b41 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:51.800950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aae73a-81fa-4486-bf0a-d62a94480c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.801663 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae73a-81fa-4486-bf0a-d62a94480c93 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.801978 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae73a-81fa-4486-bf0a-d62a94480c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.802199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae73a-81fa-4486-bf0a-d62a94480c93 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.802926 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aae73a-81fa-4486-bf0a-d62a94480c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:51.803230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aae73a-81fa-4486-bf0a-d62a94480c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:51.803679 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bd0e9-4687-4a42-9121-d3164d8fb017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:51.804598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aae73a-81fa-4486-bf0a-d62a94480c93 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:51.804883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bd0e9-4687-4a42-9121-d3164d8fb017 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.805318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bd0e9-4687-4a42-9121-d3164d8fb017 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.805724 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bd0e9-4687-4a42-9121-d3164d8fb017 None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/14a8942a978b4d92be89376cd31a1240/roles/48c24d23f41249abbb5224c3fa457f9e` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.817332 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 317/626] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:51.818534 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55d96-ad43-4b0f-b00f-40ba8f3ced21 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:51.818813 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55d96-ad43-4b0f-b00f-40ba8f3ced21 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:51.819091 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55d96-ad43-4b0f-b00f-40ba8f3ced21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:51.822223 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bd0e9-4687-4a42-9121-d3164d8fb017 None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=14a8942a978b4d92be89376cd31a1240, role_id=48c24d23f41249abbb5224c3fa457f9e)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:51.823937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bd0e9-4687-4a42-9121-d3164d8fb017 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:51.845257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 310/627] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:02:51 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/14a8942a978b4d92be89376cd31a1240/roles/48c24d23f41249abbb5224c3fa457f9e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:02:51.845257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bd0e9-4687-4a42-9121-d3164d8fb017 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:51.846330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-37b55d96-ad43-4b0f-b00f-40ba8f3ced21 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:51.847996 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-37b55d96-ad43-4b0f-b00f-40ba8f3ced21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:51.987760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 318/628] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:52.007076 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bd0e9-4687-4a42-9121-d3164d8fb017 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.028053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b55d96-ad43-4b0f-b00f-40ba8f3ced21 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.043575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.044385 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.049187 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.049187 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.066170 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:52.067960 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:52.085512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:52.086734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:02:52.087819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:52.090459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 311/629] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:02:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:52.096568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.187730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b55d96-ad43-4b0f-b00f-40ba8f3ced21 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.225371 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdd632-6490-46b7-a53b-bebbb5ff722f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.225921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0a623-e320-4f1c-8833-84736c6c4dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.226411 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0a623-e320-4f1c-8833-84736c6c4dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.226783 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0a623-e320-4f1c-8833-84736c6c4dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.227147 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0a623-e320-4f1c-8833-84736c6c4dc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.238124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0a623-e320-4f1c-8833-84736c6c4dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-d0c0a623-e320-4f1c-8833-84736c6c4dc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:52.246572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:52.247725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:52.248922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:52.255760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 319/630] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:52.256802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0a623-e320-4f1c-8833-84736c6c4dc5 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.274726 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.275672 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.276113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.276539 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.278176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:52.278642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:52.280246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:52.313292 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 312/631] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:52.319025 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.360811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0a623-e320-4f1c-8833-84736c6c4dc5 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.395884 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.396863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.397272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.397696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.398760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:52.399172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:52.400646 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:52.413164 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 320/632] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:52.415080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.422137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d1578-6297-445f-9c32-8289bdd9dc28 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.456499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bed48-ec2d-466e-84d6-12e9125125cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.457463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bed48-ec2d-466e-84d6-12e9125125cf None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.457892 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bed48-ec2d-466e-84d6-12e9125125cf None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.458312 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bed48-ec2d-466e-84d6-12e9125125cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.518512 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1975f5-505c-4b48-b565-dbd71ea999d5 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.527543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bed48-ec2d-466e-84d6-12e9125125cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:52.529294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bed48-ec2d-466e-84d6-12e9125125cf None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:52.553460 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.554392 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.554778 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.555195 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.566897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:52.568519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:52.585899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:52.587011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:52.588133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:52.588133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:52.588133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:52.588133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:52.594024 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 321/633] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:52.595278 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.633592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 313/634] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:02:52.640028 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bed48-ec2d-466e-84d6-12e9125125cf None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.702515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674259ba-47b0-4516-a846-8105e205ce14 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.739545 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.740375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.740758 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.741169 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.741998 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:52.742429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:52.743898 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:52.752209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bed48-ec2d-466e-84d6-12e9125125cf None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.761145 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 322/635] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:52.762790 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.787424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.788448 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.788879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.789406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.800438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:52.802241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:52.820612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:52.821623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:52.822531 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:52.822531 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:52.822531 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:52.822531 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:52.826241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 314/636] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:52.831710 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.869365 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7183f7b-7a57-4fea-bbb3-d716227f719d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:52.908566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7af67-a2f3-433e-9dc3-b9e8908e7dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:52.909421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7af67-a2f3-433e-9dc3-b9e8908e7dce None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.909702 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7af67-a2f3-433e-9dc3-b9e8908e7dce None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.909974 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7af67-a2f3-433e-9dc3-b9e8908e7dce None admin] PATH_INFO: `/v3/users/glance` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.919966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7af67-a2f3-433e-9dc3-b9e8908e7dce None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:52.921855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7af67-a2f3-433e-9dc3-b9e8908e7dce None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a6c7af67-a2f3-433e-9dc3-b9e8908e7dce None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:52.939999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:52.941033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:02:52.942060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:52.946638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 323/637] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:52.948221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818c74c-8ad1-49ff-97b3-44eb46ea16a9 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:52.949565 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818c74c-8ad1-49ff-97b3-44eb46ea16a9 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:52.949876 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818c74c-8ad1-49ff-97b3-44eb46ea16a9 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:52.952542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 324/638] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:52.952542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818c74c-8ad1-49ff-97b3-44eb46ea16a9 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:52.962796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ff3ce-6c0b-41ea-8327-66a9f780ecbf None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.023211 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ded6f8-2e37-4c1b-b284-23c601369348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.026765 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded6f8-2e37-4c1b-b284-23c601369348 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.027777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded6f8-2e37-4c1b-b284-23c601369348 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.028530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ded6f8-2e37-4c1b-b284-23c601369348 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.030880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ded6f8-2e37-4c1b-b284-23c601369348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:53.031540 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ded6f8-2e37-4c1b-b284-23c601369348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:53.034043 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ded6f8-2e37-4c1b-b284-23c601369348 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:53.050609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 315/639] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:53.052842 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1acf0-5ae8-4392-9905-7e9c7144dad8 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.053337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1acf0-5ae8-4392-9905-7e9c7144dad8 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.053785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1acf0-5ae8-4392-9905-7e9c7144dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.075228 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818c74c-8ad1-49ff-97b3-44eb46ea16a9 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.076037 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-84f1acf0-5ae8-4392-9905-7e9c7144dad8 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:53.078378 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-84f1acf0-5ae8-4392-9905-7e9c7144dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:53.110803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.111605 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.111914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.112259 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.113225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:53.113604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:53.115117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:53.137797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 325/640] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:02:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:53.138940 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.165153 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 316/641] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:53.165153 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1acf0-5ae8-4392-9905-7e9c7144dad8 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.251129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360fed9-ba2e-4e44-938d-e2608d72aedb None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.261672 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1acf0-5ae8-4392-9905-7e9c7144dad8 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.284662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713ad69-3e58-41ab-97e2-bd962e783f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.285462 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713ad69-3e58-41ab-97e2-bd962e783f34 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.285746 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713ad69-3e58-41ab-97e2-bd962e783f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.286013 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713ad69-3e58-41ab-97e2-bd962e783f34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.292962 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713ad69-3e58-41ab-97e2-bd962e783f34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:53.293794 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1101f-29cc-404f-a678-d38ae106b324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.294439 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1101f-29cc-404f-a678-d38ae106b324 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.294555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713ad69-3e58-41ab-97e2-bd962e783f34 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:53.294715 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1101f-29cc-404f-a678-d38ae106b324 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.294982 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1101f-29cc-404f-a678-d38ae106b324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-8713ad69-3e58-41ab-97e2-bd962e783f34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:53.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:53.312776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:53.313811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:53.313811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:53.313811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:53.313811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:53.313811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:53.313811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:53.313811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:53.313811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:53.315715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 326/642] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:53.317088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590254e-3c4c-4757-9669-d277ca26ae1e None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.317522 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590254e-3c4c-4757-9669-d277ca26ae1e None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.318882 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590254e-3c4c-4757-9669-d277ca26ae1e None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.320216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 327/643] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:53.320970 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590254e-3c4c-4757-9669-d277ca26ae1e None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.371892 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1101f-29cc-404f-a678-d38ae106b324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:53.373445 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1101f-29cc-404f-a678-d38ae106b324 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:53.430213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590254e-3c4c-4757-9669-d277ca26ae1e None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.465553 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1059e-bd65-4d02-82b4-897ae0b7603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.466340 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1059e-bd65-4d02-82b4-897ae0b7603e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.466621 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1059e-bd65-4d02-82b4-897ae0b7603e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.466892 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1059e-bd65-4d02-82b4-897ae0b7603e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.476797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1059e-bd65-4d02-82b4-897ae0b7603e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-74c1059e-bd65-4d02-82b4-897ae0b7603e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:53.485016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:53.486011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:53.486901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:53.487562 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 317/644] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:53.487634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 328/645] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:53.487634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcabb9d-ba28-4039-aeed-4afc0247cff6 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.488260 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcabb9d-ba28-4039-aeed-4afc0247cff6 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.488260 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcabb9d-ba28-4039-aeed-4afc0247cff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.491300 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1101f-29cc-404f-a678-d38ae106b324 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.505539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-ebcabb9d-ba28-4039-aeed-4afc0247cff6 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:53.506742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-ebcabb9d-ba28-4039-aeed-4afc0247cff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:53.581132 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 329/646] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:53.581132 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcabb9d-ba28-4039-aeed-4afc0247cff6 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.596311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1101f-29cc-404f-a678-d38ae106b324 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.629762 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.630415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.630703 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.630962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.631753 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:53.632063 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:53.633510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:53.649351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 318/647] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:53.650581 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.682539 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcabb9d-ba28-4039-aeed-4afc0247cff6 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.713863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.715183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.715459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.715728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.725835 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:53.727359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:53.744825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:53.745736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:53.751379 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:53.751379 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:53.751379 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:53.751379 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:53.751379 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 330/648] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:53.751666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f11bf1-87bd-41b7-944b-851949bb286e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.751738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.780010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.780670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.780952 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.781269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.782018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:53.782325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:53.783707 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:53.791303 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 319/649] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:53.792074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.852118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84c39-cdbf-47f2-b0d3-5ec1c8786b52 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.885634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d381e-3419-487d-80c5-cf706667f321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.886959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d381e-3419-487d-80c5-cf706667f321 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.887235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d381e-3419-487d-80c5-cf706667f321 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.887504 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d381e-3419-487d-80c5-cf706667f321 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.896000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf9968-722b-4d5b-b3f8-2b9e5d5b05f5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:53.897383 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d381e-3419-487d-80c5-cf706667f321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a19d381e-3419-487d-80c5-cf706667f321 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:53.905222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:53.906167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:53.907139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 331/650] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:53.909010 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d381e-3419-487d-80c5-cf706667f321 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:53.936296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:53.936951 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:53.937261 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:53.937535 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:53.938358 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=512c115af57c463eb1fbe636667306dd) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:53.938668 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:53.940090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:53.957404 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 320/651] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/role_assignments?user.id=512c115af57c463eb1fbe636667306dd&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:53.957404 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.028393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d381e-3419-487d-80c5-cf706667f321 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.066965 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.067792 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.068177 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.068554 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.069464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:54.069944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.071523 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.085189 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 332/652] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:54.086328 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.094778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1a000-7deb-445c-8e66-614a631f2ed2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.151204 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.153152 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.153152 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.153324 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.164858 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.166350 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:54.184799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:54.185761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:54.186642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:54.186642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:54.186642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:54.186642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:54.186642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 321/653] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:54.186642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.231193 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89191f7b-6d78-4387-938c-4a92c7657890 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.274651 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.275318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.275599 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.275874 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.276741 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:54.277573 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.278963 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.287130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 333/654] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:54.288113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.312520 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273478dd-04d6-4e28-bbef-2672c257deb6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.361316 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.362331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.362615 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.362950 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.375834 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.377564 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:54.392911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:54.394046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:54.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:54.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:54.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:54.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:54.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:54.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:54.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:54.395113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:54.397387 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 322/655] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:54.402688 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.402845 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68b16d-3731-48f1-9779-28143560124e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.445058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.445931 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.446312 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.446922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.447513 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:54.447940 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.450008 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.462291 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 334/656] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:54.463318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.506418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e78de1-9755-49cd-82f6-0fd6f3d040b6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.540459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.541274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.541577 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.541855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.552041 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.553509 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.563649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87451a8-b1d5-4f20-a1b7-0449b315359e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:54.570130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:54.571054 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:54.571973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:54.571973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:54.571973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:54.571973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:54.571973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 323/657] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:54.572680 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.604541 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405cbea-22f1-449d-a430-26fe6948d5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.609608 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405cbea-22f1-449d-a430-26fe6948d5ae None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.611503 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405cbea-22f1-449d-a430-26fe6948d5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.611976 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405cbea-22f1-449d-a430-26fe6948d5ae None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.613720 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405cbea-22f1-449d-a430-26fe6948d5ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:54.614189 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405cbea-22f1-449d-a430-26fe6948d5ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405cbea-22f1-449d-a430-26fe6948d5ae None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.633486 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 335/658] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:54.634977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1eb23-2dff-430f-a40c-ce7d9cd7e06d None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.635268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1eb23-2dff-430f-a40c-ce7d9cd7e06d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.635545 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1eb23-2dff-430f-a40c-ce7d9cd7e06d None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.637559 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 336/659] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:54.642137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1eb23-2dff-430f-a40c-ce7d9cd7e06d None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.687537 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23e918-6f76-47de-9700-d79edbdf5b6e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.721603 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.722389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.722664 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.722928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.729846 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.731281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.744330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1eb23-2dff-430f-a40c-ce7d9cd7e06d None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:54.745238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:54.746282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 324/660] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:54.747270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.783367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0ff0a-4952-4a59-8022-a83fe4478c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.784038 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0ff0a-4952-4a59-8022-a83fe4478c40 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.784313 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0ff0a-4952-4a59-8022-a83fe4478c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.784589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0ff0a-4952-4a59-8022-a83fe4478c40 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.785359 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0ff0a-4952-4a59-8022-a83fe4478c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:54.785667 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0ff0a-4952-4a59-8022-a83fe4478c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.787045 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0ff0a-4952-4a59-8022-a83fe4478c40 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.803803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 337/661] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:54.803803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085207c-c479-449d-a68a-7394da78ea4f None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.803803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085207c-c479-449d-a68a-7394da78ea4f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.803803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085207c-c479-449d-a68a-7394da78ea4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.818017 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-f085207c-c479-449d-a68a-7394da78ea4f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:54.819175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-f085207c-c479-449d-a68a-7394da78ea4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:54.848509 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cfe15-e49e-4e1e-9f2e-70f4b7cb72d8 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:54.882222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:54.883098 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:54.883444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 338/662] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:54.883761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:54.884165 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:54.884262 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085207c-c479-449d-a68a-7394da78ea4f None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:54.885244 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=489243fe9de94fc693a06aa6f3183e18) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:54.885553 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:54.887001 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:54.897642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 325/663] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/role_assignments?user.id=489243fe9de94fc693a06aa6f3183e18&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:54.898554 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.023537 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26724f-b65d-4827-b907-5123934e783c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.024647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085207c-c479-449d-a68a-7394da78ea4f None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.062355 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.062882 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.063125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.063309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.063613 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.064482 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:55.064598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.064787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.065269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.065706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.066275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.074264 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.075982 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.089828 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 339/664] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:55.090756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:55.092962 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:55.094093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:02:55.095222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:55.097689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 326/665] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:55.098696 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.223039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca5fae-0ccd-4f4d-aede-cbae8a42715a None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.225920 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6b806-647f-4646-a62a-bfd543462d43 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.264170 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443e2bc-0398-4cd1-ac0c-d79298fb5d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.264958 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e2bc-0398-4cd1-ac0c-d79298fb5d88 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.265201 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ce4b8-6742-41d8-9b5f-ee37fbb712d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.265339 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e2bc-0398-4cd1-ac0c-d79298fb5d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.265544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443e2bc-0398-4cd1-ac0c-d79298fb5d88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.265824 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ce4b8-6742-41d8-9b5f-ee37fbb712d4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.266108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ce4b8-6742-41d8-9b5f-ee37fbb712d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.266384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ce4b8-6742-41d8-9b5f-ee37fbb712d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.278784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443e2bc-0398-4cd1-ac0c-d79298fb5d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-5443e2bc-0398-4cd1-ac0c-d79298fb5d88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:55.286591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:55.287546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:02:55.288400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:55.293131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 340/666] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:55.294176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443e2bc-0398-4cd1-ac0c-d79298fb5d88 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.350339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ce4b8-6742-41d8-9b5f-ee37fbb712d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.351798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ce4b8-6742-41d8-9b5f-ee37fbb712d4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.399408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443e2bc-0398-4cd1-ac0c-d79298fb5d88 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.434607 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.435700 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.436230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.436717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.437790 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:55.438303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.439960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.458689 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 341/667] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:55.459935 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.473937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 327/668] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:55.474827 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ce4b8-6742-41d8-9b5f-ee37fbb712d4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.560984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388a713-7c10-477e-9634-4660e40bcb1f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.571366 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ce4b8-6742-41d8-9b5f-ee37fbb712d4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.595851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.597012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.597554 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.598026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.599015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:55.599521 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.601154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.608949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd916a-bf39-4548-9c87-91cb8bf013e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.608949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd916a-bf39-4548-9c87-91cb8bf013e5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.608949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd916a-bf39-4548-9c87-91cb8bf013e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.608949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd916a-bf39-4548-9c87-91cb8bf013e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.610627 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 342/669] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:55.611445 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.692754 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd916a-bf39-4548-9c87-91cb8bf013e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.694378 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd916a-bf39-4548-9c87-91cb8bf013e5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.714337 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2a98e-a040-4071-bc21-6283a8fb992e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.748762 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.749528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.749812 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.750085 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.760705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.762777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:55.783588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:55.784552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:55.785457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:55.785457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:55.785457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:55.785457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:55.789119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 343/670] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:55.789948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.821217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 328/671] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:55.822099 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd916a-bf39-4548-9c87-91cb8bf013e5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.917610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ad128-7fb5-46c6-8482-410702dead9d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.921667 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd916a-bf39-4548-9c87-91cb8bf013e5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:55.957967 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.958788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.958965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:55.959162 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.959314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.959529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:55.959815 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:55.960089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:55.960822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:55.961154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.963048 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.970820 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:55.972277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:55.981208 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 329/672] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:55.982107 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:55.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:55.994108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:55.994997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:55.994997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:55.994997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:55.994997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:55.994997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 344/673] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:55.995723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.093645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12182b48-0785-439c-86a0-e42fdff9ea62 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.108550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb123de-5d5e-4b2a-8a17-2e237666b4c5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.139965 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.140760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.141064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.141341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.150190 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.151662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.156924 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.158247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.158729 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.159231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:56.166080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:56.166966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:56.168428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:56.168428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:56.168428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:56.168428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:56.168428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:56.168428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:56.168428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:56.168428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:56.172126 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 345/674] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:56.172977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.174210 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.176082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:56.197106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:56.198085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:56.199024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:56.199024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:56.199024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:56.199024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:56.203250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 330/675] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:56.204263 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.295108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b6046-4718-4b36-b0b9-59d32c049cf9 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.307281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3dc1-9247-4120-862e-c89ba800647d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.330529 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.331541 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.332022 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.332518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.333495 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:56.334073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.335646 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.342899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.343616 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.343956 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.344175 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.345088 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:56.345330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.346771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.350844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 346/676] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:56.351900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.365160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 331/677] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:56.366139 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.456571 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d152f9a-e84b-4d5f-a67f-680688c06fde None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.466385 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ef647-f128-4cac-995c-945c791fa319 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.488632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.489313 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.489596 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.489869 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.490585 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:56.490890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.492241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.500493 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58602000-ccfc-41bd-bd70-83d4cd09eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.501307 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58602000-ccfc-41bd-bd70-83d4cd09eb5d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.501589 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58602000-ccfc-41bd-bd70-83d4cd09eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.501863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58602000-ccfc-41bd-bd70-83d4cd09eb5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.509118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 347/678] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:56.509118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.509348 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58602000-ccfc-41bd-bd70-83d4cd09eb5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.510642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58602000-ccfc-41bd-bd70-83d4cd09eb5d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-58602000-ccfc-41bd-bd70-83d4cd09eb5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:56.525767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:56.527406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:56.528376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 332/679] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:56.528998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d904202-8068-4702-9845-6ea42ba7963d None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.528998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d904202-8068-4702-9845-6ea42ba7963d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.529787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d904202-8068-4702-9845-6ea42ba7963d None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.533072 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 333/680] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:56.533072 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d904202-8068-4702-9845-6ea42ba7963d None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.610180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede9d1-8bc5-42fa-8b01-998be48f8403 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.631998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d904202-8068-4702-9845-6ea42ba7963d None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.644049 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.644694 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.644995 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.645298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.646123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=796bb66e8c14434097fde50e6815c9cd) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:56.646433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.647856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.661098 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 348/681] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/role_assignments?user.id=796bb66e8c14434097fde50e6815c9cd&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:56.662060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.672031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbf926-08e9-4f1b-939a-25bdbea9082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.672941 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf926-08e9-4f1b-939a-25bdbea9082d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.673335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf926-08e9-4f1b-939a-25bdbea9082d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.673682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf926-08e9-4f1b-939a-25bdbea9082d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.684905 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbf926-08e9-4f1b-939a-25bdbea9082d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.687129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbf926-08e9-4f1b-939a-25bdbea9082d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-42bbf926-08e9-4f1b-939a-25bdbea9082d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:56.706633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:56.707619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:56.708552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:56.708552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:56.708552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:56.708552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:56.708552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:56.708552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:56.708552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:56.708552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:56.713640 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 334/682] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:56.715222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eb294-d884-4aaa-acc0-1fa539c70a44 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.715510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eb294-d884-4aaa-acc0-1fa539c70a44 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.715790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eb294-d884-4aaa-acc0-1fa539c70a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.741532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-7c1eb294-d884-4aaa-acc0-1fa539c70a44 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:56.742954 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-7c1eb294-d884-4aaa-acc0-1fa539c70a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:56.793287 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e91f-759e-42eb-b382-38ef3ca83370 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.818896 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 335/683] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:56.820090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eb294-d884-4aaa-acc0-1fa539c70a44 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.851029 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.851964 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.852413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.852871 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.853892 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:56.854349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.856239 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:56.875669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 349/684] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:56.876191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:56.946021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eb294-d884-4aaa-acc0-1fa539c70a44 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:56.983530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:56.984242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:56.984566 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:56.984838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:56.985658 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:56.985963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:56.987372 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.001291 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 336/685] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:57.002152 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.010259 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58b254-5c74-49b0-88eb-2d86da14e0d8 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.046738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160436b-2240-4034-9b7c-0a640045ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.047524 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160436b-2240-4034-9b7c-0a640045ab78 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.047857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160436b-2240-4034-9b7c-0a640045ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.048129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160436b-2240-4034-9b7c-0a640045ab78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.059132 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160436b-2240-4034-9b7c-0a640045ab78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-2160436b-2240-4034-9b7c-0a640045ab78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:02:57.066662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:57.067601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:57.068511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 350/686] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:57.070021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160436b-2240-4034-9b7c-0a640045ab78 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.118157 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8edc8-ac98-41a2-a03b-b4491f75232a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.155089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7cd9d-9d7a-42fa-bf32-212bfeece7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.156099 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7cd9d-9d7a-42fa-bf32-212bfeece7de None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.156583 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7cd9d-9d7a-42fa-bf32-212bfeece7de None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.157087 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7cd9d-9d7a-42fa-bf32-212bfeece7de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.187789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160436b-2240-4034-9b7c-0a640045ab78 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.226207 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.227234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.227234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.227425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.228262 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=adf51a2d99154eccbc74f0c63fc18bf9) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:57.228570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:57.230721 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.237804 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 351/687] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/role_assignments?user.id=adf51a2d99154eccbc74f0c63fc18bf9&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:57.238669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.243806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7cd9d-9d7a-42fa-bf32-212bfeece7de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:57.245827 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7cd9d-9d7a-42fa-bf32-212bfeece7de None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.345894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69657fe-d03b-41b1-ada3-cf467e3c8625 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.352721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 337/688] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:57.366146 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7cd9d-9d7a-42fa-bf32-212bfeece7de None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.394753 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3f0e3-560c-40d8-a989-c4ab132b1c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.395363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3f0e3-560c-40d8-a989-c4ab132b1c5f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.395642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3f0e3-560c-40d8-a989-c4ab132b1c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.395912 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3f0e3-560c-40d8-a989-c4ab132b1c5f None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.396699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3f0e3-560c-40d8-a989-c4ab132b1c5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:57.397009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3f0e3-560c-40d8-a989-c4ab132b1c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:57.400353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3f0e3-560c-40d8-a989-c4ab132b1c5f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.415221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 352/689] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:57.416588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a5ac5-7e38-4a93-9688-5da855627186 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.416870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a5ac5-7e38-4a93-9688-5da855627186 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.417750 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a5ac5-7e38-4a93-9688-5da855627186 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.419360 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 353/690] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:57.422155 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a5ac5-7e38-4a93-9688-5da855627186 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.505866 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7cd9d-9d7a-42fa-bf32-212bfeece7de None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.544334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a5ac5-7e38-4a93-9688-5da855627186 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.547011 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78bcc9-8b66-4841-8c3b-7ebd477fcc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.547796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78bcc9-8b66-4841-8c3b-7ebd477fcc39 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.548071 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78bcc9-8b66-4841-8c3b-7ebd477fcc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.548335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78bcc9-8b66-4841-8c3b-7ebd477fcc39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.562096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78bcc9-8b66-4841-8c3b-7ebd477fcc39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:57.563527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78bcc9-8b66-4841-8c3b-7ebd477fcc39 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-ae78bcc9-8b66-4841-8c3b-7ebd477fcc39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:57.580921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:57.581875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:57.582793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:57.582793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:57.582793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:57.582793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:57.582793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 338/691] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:57.583816 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f882e-2b7e-4beb-9737-f495c3d52152 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.584092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f882e-2b7e-4beb-9737-f495c3d52152 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.584367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f882e-2b7e-4beb-9737-f495c3d52152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.592084 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.592084 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.592084 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.592084 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.603813 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:57.604118 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-9a3f882e-2b7e-4beb-9737-f495c3d52152 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:57.605928 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.606203 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-9a3f882e-2b7e-4beb-9737-f495c3d52152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:57.623527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:57.624623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:57.625708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:57.625708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:57.625708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:57.625708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:57.625708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 354/692] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:57.625978 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.681093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 339/693] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:57.681093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f882e-2b7e-4beb-9737-f495c3d52152 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.751497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528839a-67e8-4400-97b2-23f95e05467c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.785961 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.786817 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.786951 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.787133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.787851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:57.788213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:57.789550 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.807237 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 355/694] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:57.809663 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.811384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f882e-2b7e-4beb-9737-f495c3d52152 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.849528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.850210 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.850491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.850763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.851482 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:57.851782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:57.853155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.867316 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 340/695] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:57.868136 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:57.929297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81aa7a-d750-4b45-8062-fb4b66db2cdd None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.965737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:57.966453 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:57.966730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:57.967010 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:57.974548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634916ab-461a-432a-8a3e-90373fcbf133 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:57.977184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:57.978640 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:57.998154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:57.999105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:58.000000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:58.000000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:58.000000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:58.000000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:58.004370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 356/696] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:58.005225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:58.018593 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:58.019864 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:58.020680 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:58.021423 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:58.030865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:58.033267 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:58.055479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:58.056430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:58.057361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:58.057361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:58.057361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:58.057361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:58.057361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:58.057361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:58.057361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:58.057361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:58.063336 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 341/697] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:58.064250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:58.126623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309b09f-00bf-4aff-913e-8dfb57557280 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:58.169244 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af655249-5e9c-4dfe-a140-241e51947f3e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:58.170293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:58.172235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:58.172717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:58.173224 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:58.182216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:58.184690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:58.203709 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:58.204361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:58.204636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:58.204902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:58.205666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:58.205966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:02:58.207068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:58.208333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:02:58.209385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:58.209917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:58.210738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 357/698] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:02:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:58.211621 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:58.224127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 342/699] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:58.225542 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:58.314626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7294-5054-4f87-8aac-e2fe8ebe04f9 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:58.341848 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d46c2c-e96f-4ec8-b696-7fad4e8540d6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:58.356790 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:58.357859 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:58.358585 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:58.359076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:58.360049 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:58.360549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:58.362846 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:58.374920 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:58.375408 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:58.375601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:58.375796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:58.376624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:58.376971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:58.378359 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:58.380007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 358/700] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:58.381183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:58.403298 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 343/701] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:02:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:58.404508 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:58.493858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b7cb7-1be6-4337-af69-abf27f23b775 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:58.518133 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c505612-2b37-4a57-a831-6bdfb787f4e8 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:58.536572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c066a-a183-4275-b9cc-974815ad6789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:58.537599 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c066a-a183-4275-b9cc-974815ad6789 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:58.538078 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c066a-a183-4275-b9cc-974815ad6789 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:58.538941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c066a-a183-4275-b9cc-974815ad6789 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:58.539977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c066a-a183-4275-b9cc-974815ad6789 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:58.541600 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c066a-a183-4275-b9cc-974815ad6789 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:58.553646 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:58.554510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:58.554911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:58.555234 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:58.556491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=14a8942a978b4d92be89376cd31a1240) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:58.556869 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:58.559207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:58.565661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 359/702] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:02:58 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:02:58.566725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c066a-a183-4275-b9cc-974815ad6789 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:58.568856 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 344/703] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:58 2022] GET /identity/v3/role_assignments?user.id=14a8942a978b4d92be89376cd31a1240&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:58.744157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c066a-a183-4275-b9cc-974815ad6789 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:58.786858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c413e-2a4b-4958-be3a-796ef5377aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:58.787515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c413e-2a4b-4958-be3a-796ef5377aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:58.787790 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c413e-2a4b-4958-be3a-796ef5377aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:58.788060 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c413e-2a4b-4958-be3a-796ef5377aa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:58.864637 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c413e-2a4b-4958-be3a-796ef5377aa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:58.866160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c413e-2a4b-4958-be3a-796ef5377aa2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:58.976418 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 360/704] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:02:58.981691 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:59.093950 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd97d62-801c-4283-b1c3-a56a9096490e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:59.133075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:59.133869 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.134154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.134430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.148992 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:59.150514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:59.161114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48d712-351a-49d7-a134-e2707912e399 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.161698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48d712-351a-49d7-a134-e2707912e399 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.162169 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48d712-351a-49d7-a134-e2707912e399 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.164142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 361/705] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:59.170067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f08c2-e06a-445d-8759-7a70a2415aa0 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.170356 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f08c2-e06a-445d-8759-7a70a2415aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.170638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f08c2-e06a-445d-8759-7a70a2415aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:02:59.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:59.174159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:59.175074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:02:59.175074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:02:59.175074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:02:59.175074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:59.175074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 345/706] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:59.184918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:59.200855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-a18f08c2-e06a-445d-8759-7a70a2415aa0 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:59.202090 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-a18f08c2-e06a-445d-8759-7a70a2415aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:02:59.302027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 362/707] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:02:59.333410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b325dc-dee0-4df8-8a67-04cf5bf4ffc4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:59.336988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f08c2-e06a-445d-8759-7a70a2415aa0 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:59.375655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:59.376241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.376526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.376801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.377561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:59.377869 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:59.379497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:59.394408 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 346/708] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:59.402449 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:59.463765 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f08c2-e06a-445d-8759-7a70a2415aa0 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:59.502106 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:59.503228 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.503716 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.504434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.505319 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:59.506875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:59.522206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:59.523205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:02:59.524064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:02:59.527141 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 363/709] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:02:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:02:59.532942 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:59.545647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468c562-8ca6-4474-b1d0-bc8ab13bef8e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:59.618376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:59.619512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.619974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.620961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.635629 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:59.638560 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:02:59.667880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:02:59.668969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:02:59.669902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:02:59.669902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:02:59.669902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:02:59.669902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:02:59.669902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:02:59.669902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:02:59.669902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:02:59.669902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:02:59.678880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 347/710] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:02:59.687111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:59.723439 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327a6d-6f6a-4a82-b988-832ee59b305e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:59.765634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:59.766551 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.768326 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.768753 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.770597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:59.771737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:59.775587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:59.784915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 364/711] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:02:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:59.792990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:59.855232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83c602-5f2f-4ec4-9680-7cd3109e4f54 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:59.892706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf68a-54b3-46ae-971b-b5935caba666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:59.893861 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf68a-54b3-46ae-971b-b5935caba666 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.893861 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf68a-54b3-46ae-971b-b5935caba666 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.894222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf68a-54b3-46ae-971b-b5935caba666 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.895119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf68a-54b3-46ae-971b-b5935caba666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:59.895522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf68a-54b3-46ae-971b-b5935caba666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:59.897299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf68a-54b3-46ae-971b-b5935caba666 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:59.913819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 348/712] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:02:59.915205 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec97984-c2d7-4f2d-8e29-ec81254d5af5 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.915743 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec97984-c2d7-4f2d-8e29-ec81254d5af5 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.915994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec97984-c2d7-4f2d-8e29-ec81254d5af5 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.918135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 349/713] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:02:59.918947 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec97984-c2d7-4f2d-8e29-ec81254d5af5 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:02:59.926376 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd89b05-2b42-45e7-a6fe-13ae53e91f71 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:02:59.961612 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaf197-d957-4a29-8b00-432b707ae643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:02:59.962273 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaf197-d957-4a29-8b00-432b707ae643 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.962549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaf197-d957-4a29-8b00-432b707ae643 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.962816 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaf197-d957-4a29-8b00-432b707ae643 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.963666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaf197-d957-4a29-8b00-432b707ae643 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:02:59.963786 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaf197-d957-4a29-8b00-432b707ae643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:02:59.965816 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaf197-d957-4a29-8b00-432b707ae643 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:02:59.971606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 365/714] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:02:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:02:59.972925 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe88cdf-8dbf-4be9-a134-d79b29335d7f None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:02:59.973234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe88cdf-8dbf-4be9-a134-d79b29335d7f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:02:59.973512 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe88cdf-8dbf-4be9-a134-d79b29335d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:02:59.991764 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-bbe88cdf-8dbf-4be9-a134-d79b29335d7f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:02:59.993074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-bbe88cdf-8dbf-4be9-a134-d79b29335d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:00.031070 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec97984-c2d7-4f2d-8e29-ec81254d5af5 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:00.081199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 366/715] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:03:00.097899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:00.098982 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:00.099474 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:00.099596 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe88cdf-8dbf-4be9-a134-d79b29335d7f None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:00.100093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:00.101197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=512c115af57c463eb1fbe636667306dd) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:00.101730 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:00.103419 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:00.114543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 350/716] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:02:59 2022] GET /identity/v3/role_assignments?user.id=512c115af57c463eb1fbe636667306dd&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:00.231151 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe88cdf-8dbf-4be9-a134-d79b29335d7f None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:00.270559 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:00.271370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:00.271702 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:00.271976 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] PATH_INFO: `/v3/services/network` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:00.272632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:00.274109 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:00.287296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:00.288385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:00.289271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:00.293202 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 367/717] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:00 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:00.295747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:00.424788 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cd5f0-9345-4114-8bf9-af694b2c5988 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:00.469714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011594dc-cfb1-472c-9bbf-f8411b50b485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:00.470383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011594dc-cfb1-472c-9bbf-f8411b50b485 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:00.470662 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011594dc-cfb1-472c-9bbf-f8411b50b485 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:00.470936 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011594dc-cfb1-472c-9bbf-f8411b50b485 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:00.471650 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011594dc-cfb1-472c-9bbf-f8411b50b485 None admin] RBAC: Adding query filter params (name=network) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:00.471964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011594dc-cfb1-472c-9bbf-f8411b50b485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:00.473353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011594dc-cfb1-472c-9bbf-f8411b50b485 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:00.481511 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 351/718] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:00.486956 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:00.610661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b3ccb-96d9-4954-ac39-5996fcc1e72b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:00.648001 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7f23c-59a4-4d53-a7a2-74edde426896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:00.649000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f23c-59a4-4d53-a7a2-74edde426896 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:00.649497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f23c-59a4-4d53-a7a2-74edde426896 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:00.649962 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f23c-59a4-4d53-a7a2-74edde426896 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:00.650879 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7f23c-59a4-4d53-a7a2-74edde426896 None admin] RBAC: Adding query filter params (type=network) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:00.651448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7f23c-59a4-4d53-a7a2-74edde426896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:00.653206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7f23c-59a4-4d53-a7a2-74edde426896 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:00.658182 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 368/719] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:01.363839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5033fac-88ec-42b5-bdac-3da17c04a479 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:01.364127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5033fac-88ec-42b5-bdac-3da17c04a479 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:01.364410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5033fac-88ec-42b5-bdac-3da17c04a479 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:01.366032 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 352/720] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:01.372046 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ad30-65b3-4dd0-a399-013b8e654dda None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:01.372330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ad30-65b3-4dd0-a399-013b8e654dda None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:01.372602 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ad30-65b3-4dd0-a399-013b8e654dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:01.397168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-5532ad30-65b3-4dd0-a399-013b8e654dda None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:01.398344 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-5532ad30-65b3-4dd0-a399-013b8e654dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:01.478585 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 369/721] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:03:01.514227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5033fac-88ec-42b5-bdac-3da17c04a479 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:01.638712 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5033fac-88ec-42b5-bdac-3da17c04a479 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:01.686406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecc60f-4127-4119-9b08-1bb38fb332b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:01.687126 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc60f-4127-4119-9b08-1bb38fb332b7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:01.687400 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc60f-4127-4119-9b08-1bb38fb332b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:01.687667 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecc60f-4127-4119-9b08-1bb38fb332b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:01.701838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecc60f-4127-4119-9b08-1bb38fb332b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-50ecc60f-4127-4119-9b08-1bb38fb332b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:01.709512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:01.710467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:01.711341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 353/722] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:03:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:01.719533 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ad30-65b3-4dd0-a399-013b8e654dda None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:01.772457 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091a22-2d3d-4bb2-be3c-ec2b3d70018b None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:01.772775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091a22-2d3d-4bb2-be3c-ec2b3d70018b None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:01.772999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091a22-2d3d-4bb2-be3c-ec2b3d70018b None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:01.774406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 354/723] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:01.790427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45663-f146-4237-adcf-89ecbff9f5e9 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:01.790730 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45663-f146-4237-adcf-89ecbff9f5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:01.790971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45663-f146-4237-adcf-89ecbff9f5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:01.830012 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-82e45663-f146-4237-adcf-89ecbff9f5e9 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:01.831192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-82e45663-f146-4237-adcf-89ecbff9f5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:01.836355 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ad30-65b3-4dd0-a399-013b8e654dda None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:01.879278 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:01.879946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:01.880218 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:01.880485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:01.881256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:01.881560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:01.882939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:01.904193 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 370/724] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:03:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:01.909616 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:01.942669 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 355/725] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:03:02.006325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45663-f146-4237-adcf-89ecbff9f5e9 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:02.068883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d7d06-e831-408e-8d6c-7e0152e55d40 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:02.123983 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45663-f146-4237-adcf-89ecbff9f5e9 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:02.161438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d6d22-f2cd-4d15-a171-426e7628fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:02.162232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d6d22-f2cd-4d15-a171-426e7628fde4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.162514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d6d22-f2cd-4d15-a171-426e7628fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.162785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d6d22-f2cd-4d15-a171-426e7628fde4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.172073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:02.172867 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.173177 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.173452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.182044 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d6d22-f2cd-4d15-a171-426e7628fde4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-4c2d6d22-f2cd-4d15-a171-426e7628fde4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:02.190870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:02.191805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:02.192725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 356/726] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:02.197819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d6d22-f2cd-4d15-a171-426e7628fde4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:02.199391 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:02.203905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:02.227183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:02.228194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:02.229158 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:02.229158 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:02.229158 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:02.229158 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:02.229158 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 371/727] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:02.236526 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:02.312400 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d6d22-f2cd-4d15-a171-426e7628fde4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:02.351435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:02.352491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.353065 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.353577 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.355102 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:02.355926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:02.358796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:02.373323 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 357/728] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:02.381989 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:02.455544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa3c86-7b2b-406d-8661-4ef31485ac4b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:02.491895 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f15d0-34c9-415a-8005-17865cd940f2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:02.525983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:02.527586 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.528079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.528572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.529578 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:02.532907 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:02.534712 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:02.561918 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 372/729] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:02.566736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:02.570350 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.570350 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.570350 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.571854 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:02.591009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:02.592809 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:02.619821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:02.621367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:02.622315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:02.622315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:02.622315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:02.622315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:02.631913 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 358/730] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:02.636995 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:02.749415 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4cb92-b196-476b-badc-a4fddbf40748 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:02.787790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3765a-83ce-4505-82ce-b2bd32329692 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:02.801367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea50b1-a183-4f8e-8487-aec3b7be5573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:02.803119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea50b1-a183-4f8e-8487-aec3b7be5573 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.803858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea50b1-a183-4f8e-8487-aec3b7be5573 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.804346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea50b1-a183-4f8e-8487-aec3b7be5573 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.814792 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea50b1-a183-4f8e-8487-aec3b7be5573 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:02.817079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea50b1-a183-4f8e-8487-aec3b7be5573 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:02.822581 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2e103-c717-4d36-bd9e-bcb10cfff49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:02.824045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e103-c717-4d36-bd9e-bcb10cfff49d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.824523 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e103-c717-4d36-bd9e-bcb10cfff49d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.824988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2e103-c717-4d36-bd9e-bcb10cfff49d None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.825945 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2e103-c717-4d36-bd9e-bcb10cfff49d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:02.826505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2e103-c717-4d36-bd9e-bcb10cfff49d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:02.828040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2e103-c717-4d36-bd9e-bcb10cfff49d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-e3ea50b1-a183-4f8e-8487-aec3b7be5573 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:02.834382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:02.835362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:02.836377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 373/731] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:03:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:02.837473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9113f-f0e4-400c-89a6-56871d7b4134 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.837473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9113f-f0e4-400c-89a6-56871d7b4134 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.837713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9113f-f0e4-400c-89a6-56871d7b4134 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.844270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 374/732] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:02.844270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9113f-f0e4-400c-89a6-56871d7b4134 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:02.852409 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 359/733] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:02.857460 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb67a6-364e-4706-a705-383fde058c05 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.857460 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb67a6-364e-4706-a705-383fde058c05 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.857460 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb67a6-364e-4706-a705-383fde058c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.872325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-eccb67a6-364e-4706-a705-383fde058c05 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:02.873561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-eccb67a6-364e-4706-a705-383fde058c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:02.952776 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 360/734] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:03:02.954278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb67a6-364e-4706-a705-383fde058c05 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:02.954685 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9113f-f0e4-400c-89a6-56871d7b4134 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:02.997865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012182d3-6327-4836-853e-99c98567672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:02.998533 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012182d3-6327-4836-853e-99c98567672e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:02.998813 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012182d3-6327-4836-853e-99c98567672e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:02.999086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012182d3-6327-4836-853e-99c98567672e None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:02.999974 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012182d3-6327-4836-853e-99c98567672e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:03.000238 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012182d3-6327-4836-853e-99c98567672e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.001722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012182d3-6327-4836-853e-99c98567672e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.027370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 375/735] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:03:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:03.027370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54679-696e-431d-8f2e-510ff671e21a None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.027370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54679-696e-431d-8f2e-510ff671e21a None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.027370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54679-696e-431d-8f2e-510ff671e21a None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.027370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 376/736] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:03.027370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d54679-696e-431d-8f2e-510ff671e21a None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:03.065797 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb67a6-364e-4706-a705-383fde058c05 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.105207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.106005 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.106285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.106558 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.116038 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.117508 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:03.137859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:03.138831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 361/737] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:03.139807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:03.147381 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d54679-696e-431d-8f2e-510ff671e21a None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.182066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95ba87-9b76-4651-9624-293e4741a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.183374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ba87-9b76-4651-9624-293e4741a9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.183960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ba87-9b76-4651-9624-293e4741a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.184240 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ba87-9b76-4651-9624-293e4741a9c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.196936 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95ba87-9b76-4651-9624-293e4741a9c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-de95ba87-9b76-4651-9624-293e4741a9c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:03.205061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:03.205980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:03.206836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:03.211034 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 377/738] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:03.212293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c0a5d-c3b3-4630-ad05-38fd99c73f08 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.212580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c0a5d-c3b3-4630-ad05-38fd99c73f08 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.212853 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c0a5d-c3b3-4630-ad05-38fd99c73f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.229455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-3a5c0a5d-c3b3-4630-ad05-38fd99c73f08 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:03.231172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-3a5c0a5d-c3b3-4630-ad05-38fd99c73f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:03.240433 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabf45a-cbf1-4e77-a096-71eb209ba02b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.277232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5d365-0031-40bc-b6b0-28fa9ed18219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.277814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5d365-0031-40bc-b6b0-28fa9ed18219 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.278183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5d365-0031-40bc-b6b0-28fa9ed18219 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.278467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5d365-0031-40bc-b6b0-28fa9ed18219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.301350 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 378/739] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:03:03.302189 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c0a5d-c3b3-4630-ad05-38fd99c73f08 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:03.360781 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5d365-0031-40bc-b6b0-28fa9ed18219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.362281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5d365-0031-40bc-b6b0-28fa9ed18219 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.414916 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c0a5d-c3b3-4630-ad05-38fd99c73f08 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.452638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.453345 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.453649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.453930 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.454778 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:03.455088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.456535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.457162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 362/740] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:03:03.458279 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5d365-0031-40bc-b6b0-28fa9ed18219 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:03.474590 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 379/741] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:03:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:03.475435 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:03.561342 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5d365-0031-40bc-b6b0-28fa9ed18219 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.572713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ea552-3282-4a92-9997-675b6b24cc04 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.592247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5b574-9802-4e0e-9277-981b54149b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.592901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5b574-9802-4e0e-9277-981b54149b17 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.593211 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5b574-9802-4e0e-9277-981b54149b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.593479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5b574-9802-4e0e-9277-981b54149b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.594222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5b574-9802-4e0e-9277-981b54149b17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:03.594520 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5b574-9802-4e0e-9277-981b54149b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.595879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5b574-9802-4e0e-9277-981b54149b17 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.603469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 363/742] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:03.605152 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c983a-b0cb-4061-877f-310723c31142 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.605442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c983a-b0cb-4061-877f-310723c31142 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.605711 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c983a-b0cb-4061-877f-310723c31142 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.606280 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3a64d-c946-4a7e-ac9a-17e50ab50260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.607044 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 364/743] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:03.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a64d-c946-4a7e-ac9a-17e50ab50260 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.607579 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c983a-b0cb-4061-877f-310723c31142 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:03.609382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a64d-c946-4a7e-ac9a-17e50ab50260 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.609675 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a64d-c946-4a7e-ac9a-17e50ab50260 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.610422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3a64d-c946-4a7e-ac9a-17e50ab50260 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.611784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3a64d-c946-4a7e-ac9a-17e50ab50260 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.626781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 380/744] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:03:03 2022] POST /identity/v3/services => generated 251 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:03.627665 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3a64d-c946-4a7e-ac9a-17e50ab50260 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:03.736790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c983a-b0cb-4061-877f-310723c31142 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.739185 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3a64d-c946-4a7e-ac9a-17e50ab50260 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.772253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.773088 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.773363 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.773638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.773853 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6499a70-1544-4efe-bda8-6ff12a53f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.774504 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6499a70-1544-4efe-bda8-6ff12a53f178 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.774781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6499a70-1544-4efe-bda8-6ff12a53f178 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.775051 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6499a70-1544-4efe-bda8-6ff12a53f178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.785340 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.786821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:03.804716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:03.805961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:03.808660 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:03.808660 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:03.808660 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:03.808660 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:03.808660 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 365/745] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:03.808660 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:03.846967 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6499a70-1544-4efe-bda8-6ff12a53f178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.848444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6499a70-1544-4efe-bda8-6ff12a53f178 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.910908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e9177-11f0-45a5-b2e6-2ee6f8207a93 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:03.943392 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:03.944183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.944459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.944725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.950123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 381/746] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:03:03.950123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c53fea-ffae-4e76-b332-de888fe4e660 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:03.950123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c53fea-ffae-4e76-b332-de888fe4e660 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:03.950123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c53fea-ffae-4e76-b332-de888fe4e660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:03.959434 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:03.960926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:03.963662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-94c53fea-ffae-4e76-b332-de888fe4e660 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:03.964890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-94c53fea-ffae-4e76-b332-de888fe4e660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:03.977501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:03.978401 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:03.979328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:03.979328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:03.979328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:03.979328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:03.979328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 366/747] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:03.980141 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.032460 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 382/748] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:03:04.033320 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c53fea-ffae-4e76-b332-de888fe4e660 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.086181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59232f-1f77-41d4-a7fa-b0de1a2ea23a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.127897 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.128482 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c53fea-ffae-4e76-b332-de888fe4e660 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.128615 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.129090 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.129370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.130169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:04.130378 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.131746 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.148485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 367/749] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:04.148485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.163372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.164166 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.164441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.164712 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.175408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.176851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:04.194986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:04.195950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:04.196842 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:04.196842 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:04.196842 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:04.196842 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:04.201470 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 383/750] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:04.202457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.252278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535f621-57c0-411b-af54-59fc31384348 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.290073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.290728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.291012 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.291282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.292636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:04.292949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.294351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.304947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01060a-e90a-4e33-aefb-6897f9688d89 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.305125 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 368/751] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:04.305864 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.343569 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8bc66-57ce-4287-ac3e-eed404480ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.344224 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8bc66-57ce-4287-ac3e-eed404480ced None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.344507 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8bc66-57ce-4287-ac3e-eed404480ced None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.344780 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8bc66-57ce-4287-ac3e-eed404480ced None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.345535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8bc66-57ce-4287-ac3e-eed404480ced None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.346899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8bc66-57ce-4287-ac3e-eed404480ced None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.360130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 384/752] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:03:04 2022] POST /identity/v3/services => generated 240 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:04.360929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8bc66-57ce-4287-ac3e-eed404480ced None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.421814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e0ef-84a9-4788-af75-f448d07a00de None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.464567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.465810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.466311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.466760 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.470236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8bc66-57ce-4287-ac3e-eed404480ced None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.475035 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.476823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:04.492659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:04.493689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:04.494611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:04.494611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:04.494611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:04.494611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:04.494611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:04.494611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:04.494611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:04.494611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:04.497508 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 369/753] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:04.498548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.507281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.507956 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.508233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.508508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.509394 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:04.509576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.511546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.531336 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 385/754] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:04.532279 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.639803 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea6b9a-427a-4992-8fe6-35503b9ede1a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.675968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.677016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.677486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.677889 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.686420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.688100 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:04.704018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:04.705337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:04.706490 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:04.708618 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 370/755] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:04.709540 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.763689 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa97b9-a4ac-4c91-b206-31644167b10c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.807399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.808755 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.809076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.809349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.810554 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:04.810871 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.812772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.828201 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae37e66-c4fd-4817-bfbe-a0edabbee120 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.833107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 386/756] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:04.834183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.881252 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.882071 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.882440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.882721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.892196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.893865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:04.912901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:04.914055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:04.915091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 371/757] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:04.915641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:04.951869 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafb166-db6d-4b48-b467-b6077ef370ab None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:04.992999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:04.994119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:04.994496 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:04.995073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:04.996971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:04.997445 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:04.999528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:05.021179 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 387/758] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:05.022475 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:05.045505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb66f-541e-436d-9f1f-3d0bf0ee7c2b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:05.085226 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:05.085911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:05.086190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:05.086463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:05.087293 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=c2567a28c4854d8191aa49947b0a82b8) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:05.087605 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:05.089636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:05.095059 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 372/759] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:04 2022] GET /identity/v3/role_assignments?user.id=c2567a28c4854d8191aa49947b0a82b8&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:05.095900 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:05.140269 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d69b3e-3616-47c0-bf53-8a068b5d73c6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:05.176861 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:05.177649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:05.177934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:05.178214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:05.179091 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:05.179410 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:05.180921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:05.194597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 388/760] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:05.202180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:05.203161 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb3eb8-5686-4d4c-8389-1ec1eb0e7b58 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:05.239178 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:05.239981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:05.240262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:05.240535 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:05.252524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:05.254315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:05.272422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:05.273417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:05.274328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:05.274328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:05.274328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:05.274328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:05.274328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 373/761] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:05.280811 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:05.404652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0b6b0-2808-4e10-b174-9c5fa0ab622d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:05.444560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:05.445672 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:05.446160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:05.446676 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:05.447724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=14a8942a978b4d92be89376cd31a1240) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:05.448858 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:05.457156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:05.471329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 389/762] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:05 2022] GET /identity/v3/role_assignments?user.id=14a8942a978b4d92be89376cd31a1240&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:05.555336 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5758a32-6efb-40e7-85ca-c8c57e483ad3 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:05.594210 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:05.594890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:05.595168 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:05.595440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:05.596180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:05.596487 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:05.597912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:05.610949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 374/763] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:05.616049 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:05.752180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266276f1-23ef-497c-9ce0-f6d30d37e0ea None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:05.785719 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:05.786969 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:05.787458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:05.793068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:05.806215 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:05.808570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:05.830314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:05.831275 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:05.832241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:05.832241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:05.832241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:05.832241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:05.832241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:05.832241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:05.832241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:05.832241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:05.834939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 390/764] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:05.846236 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:05.966469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639c623-eb16-4920-97f2-f0ab11c90f04 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:06.004764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270e0df-4270-4087-b25e-0a34a81b3b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:06.005384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270e0df-4270-4087-b25e-0a34a81b3b34 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:06.005663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270e0df-4270-4087-b25e-0a34a81b3b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:06.005938 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270e0df-4270-4087-b25e-0a34a81b3b34 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:06.006747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270e0df-4270-4087-b25e-0a34a81b3b34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:06.007056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270e0df-4270-4087-b25e-0a34a81b3b34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:06.008477 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270e0df-4270-4087-b25e-0a34a81b3b34 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:06.025940 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 375/765] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:06.031894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:06.043813 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943df2-8d9d-4fc6-bb3e-6ed858a02f00 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:06.044093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943df2-8d9d-4fc6-bb3e-6ed858a02f00 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:06.044357 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943df2-8d9d-4fc6-bb3e-6ed858a02f00 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:06.045992 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 376/766] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:06.052251 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596a8d-9c90-4437-93f3-82dd21c45e3a None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:06.052535 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596a8d-9c90-4437-93f3-82dd21c45e3a None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:06.052806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596a8d-9c90-4437-93f3-82dd21c45e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:06.082246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-a5596a8d-9c90-4437-93f3-82dd21c45e3a None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:06.083437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-a5596a8d-9c90-4437-93f3-82dd21c45e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:06.187132 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 377/767] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:06.221261 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd1671-2acc-49d6-b222-0d28aaae0c03 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:06.235100 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5596a8d-9c90-4437-93f3-82dd21c45e3a None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:06.272542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257232-99fc-44d7-a01d-3b57c46270e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:06.273386 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257232-99fc-44d7-a01d-3b57c46270e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:06.273670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257232-99fc-44d7-a01d-3b57c46270e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:06.273949 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a257232-99fc-44d7-a01d-3b57c46270e4 None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/512c115af57c463eb1fbe636667306dd/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:06.275932 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a257232-99fc-44d7-a01d-3b57c46270e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=512c115af57c463eb1fbe636667306dd, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:06.277949 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a257232-99fc-44d7-a01d-3b57c46270e4 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:06.303292 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 391/768] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:03:06 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/512c115af57c463eb1fbe636667306dd/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:03:06.338722 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5596a8d-9c90-4437-93f3-82dd21c45e3a None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:06.380989 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:06.381915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:06.382199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:06.382472 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:06.383154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:06.384521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:06.404277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:06.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:06.406584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 378/769] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:03:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:06.414203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257232-99fc-44d7-a01d-3b57c46270e4 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:06.526921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a257232-99fc-44d7-a01d-3b57c46270e4 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:06.562137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:06.562814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:06.563098 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:06.563374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:06.564093 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:06.564401 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:06.565790 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:06.571057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 392/770] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:03:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:06.577736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:06.684805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936db281-6042-46b3-a75b-e8dce8b9acd3 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:06.728442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:06.729140 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:06.729953 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:06.730225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:06.730939 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:06.731239 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:06.732592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:06.742180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 379/771] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:03:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:06.745672 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:06.868634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42aa41-d44b-4ad6-af51-50337fcd70ac None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:06.908719 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61453e02-214f-4a32-acbb-ff5a9246ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:06.909416 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61453e02-214f-4a32-acbb-ff5a9246ba42 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:06.909696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61453e02-214f-4a32-acbb-ff5a9246ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:06.909967 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61453e02-214f-4a32-acbb-ff5a9246ba42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:06.910769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61453e02-214f-4a32-acbb-ff5a9246ba42 None admin] RBAC: Adding query filter params (interface=public, service_id=b3c4947121894792b4d69b89d264ca2b, region_id=RegionOne) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:06.911076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61453e02-214f-4a32-acbb-ff5a9246ba42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:06.912432 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61453e02-214f-4a32-acbb-ff5a9246ba42 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:06.924877 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 393/772] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:03:06 2022] GET /identity/v3/endpoints?service_id=b3c4947121894792b4d69b89d264ca2b&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:06.936115 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:07.084778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4978c-5673-4f69-80b8-3645461d27b9 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:07.116530 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86452f-de1a-4ff0-861a-9b811d8a51e0 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:07.116530 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86452f-de1a-4ff0-861a-9b811d8a51e0 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:07.116530 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86452f-de1a-4ff0-861a-9b811d8a51e0 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:07.116530 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 394/773] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:07.125159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b06fce-6674-4b00-92b6-556851f6d69c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:07.125159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b06fce-6674-4b00-92b6-556851f6d69c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:07.125159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b06fce-6674-4b00-92b6-556851f6d69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:07.125418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbb42b-734e-4301-afe0-0da8b4b7bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:07.126016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb42b-734e-4301-afe0-0da8b4b7bec7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:07.126308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb42b-734e-4301-afe0-0da8b4b7bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:07.126583 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbb42b-734e-4301-afe0-0da8b4b7bec7 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:07.127277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbb42b-734e-4301-afe0-0da8b4b7bec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:07.128643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbb42b-734e-4301-afe0-0da8b4b7bec7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:07.136214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 380/774] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:03:06 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:07.143026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-52b06fce-6674-4b00-92b6-556851f6d69c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:07.144586 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-52b06fce-6674-4b00-92b6-556851f6d69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:07.226817 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 395/775] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:07.262452 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbb42b-734e-4301-afe0-0da8b4b7bec7 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:07.377432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbb42b-734e-4301-afe0-0da8b4b7bec7 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:07.416483 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:07.417343 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:07.417626 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:07.417894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] PATH_INFO: `/v3/services/network` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:07.418548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:07.419900 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:07.432556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:07.433617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:07.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 381/776] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:07 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:07.440237 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b06fce-6674-4b00-92b6-556851f6d69c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:07.546101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b06fce-6674-4b00-92b6-556851f6d69c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:07.580827 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:07.581874 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:07.582367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:07.582847 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:07.583763 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] RBAC: Adding query filter params (name=network) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:07.584363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:07.586021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:07.591608 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 396/777] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:07.596909 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:07.725417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac19375-60d0-42da-a4de-d8af53756bda None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:07.764459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:07.765149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:07.765429 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:07.765706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:07.766413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] RBAC: Adding query filter params (type=network) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:07.766723 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:07.768074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:07.773279 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 382/778] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:07.779561 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:07.958258 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7bbea-5e73-4d93-ad39-886c71925b77 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:07.999864 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27d543-8ceb-432f-8ab5-69a0966a7ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:08.000851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d543-8ceb-432f-8ab5-69a0966a7ede None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.005438 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d543-8ceb-432f-8ab5-69a0966a7ede None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.005716 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27d543-8ceb-432f-8ab5-69a0966a7ede None admin] PATH_INFO: `/v3/endpoints` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.006540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27d543-8ceb-432f-8ab5-69a0966a7ede None admin] RBAC: Adding query filter params (interface=public, service_id=e7602b4f548849b08edb46bd416308e1, region_id=RegionOne) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:08.006850 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27d543-8ceb-432f-8ab5-69a0966a7ede None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:08.008219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27d543-8ceb-432f-8ab5-69a0966a7ede None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:08.014222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 397/779] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:03:07 2022] GET /identity/v3/endpoints?service_id=e7602b4f548849b08edb46bd416308e1&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:08.022120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:08.094159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8fb2-fa01-4d4c-8107-8dafadadad64 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.094515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8fb2-fa01-4d4c-8107-8dafadadad64 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.094708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c8fb2-fa01-4d4c-8107-8dafadadad64 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.096400 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 398/780] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:08.110875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087b875-578e-49eb-8cfb-d0a1c9ba8f5e None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.111188 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087b875-578e-49eb-8cfb-d0a1c9ba8f5e None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.111416 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087b875-578e-49eb-8cfb-d0a1c9ba8f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.138742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-a087b875-578e-49eb-8cfb-d0a1c9ba8f5e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:08.140322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-a087b875-578e-49eb-8cfb-d0a1c9ba8f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:08.149106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeb7bb-f3fa-4282-a3a1-d4505837446a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:08.178553 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d6a74-cd91-44b6-8949-a30152b36880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:08.179215 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d6a74-cd91-44b6-8949-a30152b36880 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.179496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d6a74-cd91-44b6-8949-a30152b36880 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.179770 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d6a74-cd91-44b6-8949-a30152b36880 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.180452 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d6a74-cd91-44b6-8949-a30152b36880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:08.181838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d6a74-cd91-44b6-8949-a30152b36880 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:08.192959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 383/781] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:03:08 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:08.295617 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 399/782] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:08.344492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d6a74-cd91-44b6-8949-a30152b36880 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:08.369323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710b64-bc9c-45d1-8ed9-a96d7f1e2565 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.370540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710b64-bc9c-45d1-8ed9-a96d7f1e2565 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.371031 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710b64-bc9c-45d1-8ed9-a96d7f1e2565 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.372960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 400/783] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:08.378070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e1ac4-90e8-4e6a-a0ba-c4fc26edaaeb None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.378605 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e1ac4-90e8-4e6a-a0ba-c4fc26edaaeb None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.379375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e1ac4-90e8-4e6a-a0ba-c4fc26edaaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.407856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-962e1ac4-90e8-4e6a-a0ba-c4fc26edaaeb None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:08.409405 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-962e1ac4-90e8-4e6a-a0ba-c4fc26edaaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:08.463114 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d6a74-cd91-44b6-8949-a30152b36880 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:08.499582 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 401/784] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:08.500104 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d139b-ef6c-4227-891a-ee42eddb4c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:08.501180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d139b-ef6c-4227-891a-ee42eddb4c7b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.501682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d139b-ef6c-4227-891a-ee42eddb4c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.502021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d139b-ef6c-4227-891a-ee42eddb4c7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.515200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d139b-ef6c-4227-891a-ee42eddb4c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:08.529425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e1ac4-90e8-4e6a-a0ba-c4fc26edaaeb None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-cb9d139b-ef6c-4227-891a-ee42eddb4c7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:08.531254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:08.532190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:08.533699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 384/785] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:03:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:08.539261 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d139b-ef6c-4227-891a-ee42eddb4c7b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:08.713989 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d139b-ef6c-4227-891a-ee42eddb4c7b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:08.716968 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e1ac4-90e8-4e6a-a0ba-c4fc26edaaeb None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:08.773436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:08.774107 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.775353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09c102-c318-4ef7-902f-eb4b28dff99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:08.776173 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c102-c318-4ef7-902f-eb4b28dff99a None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:08.776457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c102-c318-4ef7-902f-eb4b28dff99a None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.776731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c102-c318-4ef7-902f-eb4b28dff99a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.777522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:08.777799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:08.778568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:08.778873 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:08.780250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:08.794484 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 385/786] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:03:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:08.798500 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09c102-c318-4ef7-902f-eb4b28dff99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-3b09c102-c318-4ef7-902f-eb4b28dff99a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:08.806546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:08.808206 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:08.809086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:08.809738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:08.818047 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 402/787] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:08.819015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09c102-c318-4ef7-902f-eb4b28dff99a None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:08.989343 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effadfa3-6289-480a-85d5-8862a55bd025 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:08.992412 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09c102-c318-4ef7-902f-eb4b28dff99a None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:09.036596 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:09.037833 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.038110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.038385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.039143 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:09.039451 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:09.043529 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:09.046753 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:09.047539 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.047817 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.048089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.064863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 403/788] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:09.066889 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:09.068356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:09.070407 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:09.101850 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:09.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:09.104225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:09.104225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:09.104225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:09.104225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:09.104225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 386/789] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:09.114169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:09.254409 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c6d7e-d390-4bcb-8678-72254e15f33b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:09.281015 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7cae-6064-4d9d-b1b0-511f27f1983d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:09.314582 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122e50e-d23f-46cc-abaf-319e05ea8a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:09.314921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27ddf3-79fa-4b92-bf43-157e6956ad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:09.315245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122e50e-d23f-46cc-abaf-319e05ea8a50 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.315527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122e50e-d23f-46cc-abaf-319e05ea8a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.315702 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ddf3-79fa-4b92-bf43-157e6956ad53 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.315825 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122e50e-d23f-46cc-abaf-319e05ea8a50 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.315978 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ddf3-79fa-4b92-bf43-157e6956ad53 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.316295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ddf3-79fa-4b92-bf43-157e6956ad53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.316511 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122e50e-d23f-46cc-abaf-319e05ea8a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:09.316846 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122e50e-d23f-46cc-abaf-319e05ea8a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:09.318338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122e50e-d23f-46cc-abaf-319e05ea8a50 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:09.332669 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 387/790] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:09.333868 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27ddf3-79fa-4b92-bf43-157e6956ad53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:09.335327 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27ddf3-79fa-4b92-bf43-157e6956ad53 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:09.336678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b99b3-de88-45dc-89ef-81c153a29de7 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.337231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b99b3-de88-45dc-89ef-81c153a29de7 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.337231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b99b3-de88-45dc-89ef-81c153a29de7 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.338507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 388/791] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:09.339761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b99b3-de88-45dc-89ef-81c153a29de7 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-ba27ddf3-79fa-4b92-bf43-157e6956ad53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:09.363026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:09.364844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:09.366285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:09.366285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:09.366285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:09.366285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:09.366285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 404/792] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:09.370107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643f36-9ffa-4511-a94f-342dbeb342c2 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.370107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643f36-9ffa-4511-a94f-342dbeb342c2 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.370107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643f36-9ffa-4511-a94f-342dbeb342c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.394168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-bc643f36-9ffa-4511-a94f-342dbeb342c2 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:09.396142 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-bc643f36-9ffa-4511-a94f-342dbeb342c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:09.487280 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 405/793] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:09.487712 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b99b3-de88-45dc-89ef-81c153a29de7 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:09.488566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc643f36-9ffa-4511-a94f-342dbeb342c2 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:09.549959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:09.550870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.551254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.551624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.568183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:09.569831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:09.593855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:09.594929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:09.595875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:09.596887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 389/794] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:03:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:09.599265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:09.683849 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc643f36-9ffa-4511-a94f-342dbeb342c2 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:09.741495 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:09.742412 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.743287 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.743725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.744610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:09.745812 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:09.747344 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:09.764048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 406/795] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:09.765259 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:09.789576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f45cd-3fae-4218-9371-8ec57801d936 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:09.860024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c14ee-4658-45bf-9fc8-42cb8f69f2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:09.860905 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c14ee-4658-45bf-9fc8-42cb8f69f2ae None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.861333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c14ee-4658-45bf-9fc8-42cb8f69f2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.861701 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c14ee-4658-45bf-9fc8-42cb8f69f2ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.878422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c14ee-4658-45bf-9fc8-42cb8f69f2ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-0d2c14ee-4658-45bf-9fc8-42cb8f69f2ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:09.893181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:09.894328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:09.895360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:09.899971 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 390/796] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:09.904190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c14ee-4658-45bf-9fc8-42cb8f69f2ae None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:09.947315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef3760-0ad8-4c87-a4c4-f3e20cf2b8c6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:09.989736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:09.990556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:09.991096 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:09.991471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:09.992992 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:09.993561 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:09.995699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.018461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 407/797] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:03:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:10.018461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.037534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c14ee-4658-45bf-9fc8-42cb8f69f2ae None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.069592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510dfb6-b76f-4af5-98bd-276b111314e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.070432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510dfb6-b76f-4af5-98bd-276b111314e8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.070708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510dfb6-b76f-4af5-98bd-276b111314e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.070972 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510dfb6-b76f-4af5-98bd-276b111314e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.079121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510dfb6-b76f-4af5-98bd-276b111314e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.080548 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510dfb6-b76f-4af5-98bd-276b111314e8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f510dfb6-b76f-4af5-98bd-276b111314e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:10.094281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:10.095246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:10.096191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 391/798] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:10.096851 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f12e2-a98b-4ee7-8927-d55733a66abe None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.096919 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f12e2-a98b-4ee7-8927-d55733a66abe None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.097541 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f12e2-a98b-4ee7-8927-d55733a66abe None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.101111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 392/799] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:10.101111 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f12e2-a98b-4ee7-8927-d55733a66abe None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.123239 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569af03-7c96-43e0-adc0-43ff0acb8084 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.158826 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.159485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.159766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.160043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.160788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:10.161128 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.162485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.171110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 408/800] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:10.172132 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.213556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f12e2-a98b-4ee7-8927-d55733a66abe None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.247273 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273010b-da01-406e-a723-be8d50173556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.248057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273010b-da01-406e-a723-be8d50173556 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.248330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273010b-da01-406e-a723-be8d50173556 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.248600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273010b-da01-406e-a723-be8d50173556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.259110 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273010b-da01-406e-a723-be8d50173556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.260575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273010b-da01-406e-a723-be8d50173556 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.272871 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501ef0e-cfba-4584-a89e-5593a077ac8c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-8273010b-da01-406e-a723-be8d50173556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:10.276821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:10.277899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:10.278852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:10.278852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:10.278852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:10.278852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:10.278852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 393/801] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:10.279486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9111ba-90c7-4aa1-80d5-61a3e46d244a None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.279764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9111ba-90c7-4aa1-80d5-61a3e46d244a None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.280031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9111ba-90c7-4aa1-80d5-61a3e46d244a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.297767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-dc9111ba-90c7-4aa1-80d5-61a3e46d244a None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:10.298898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-dc9111ba-90c7-4aa1-80d5-61a3e46d244a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:10.310685 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.311558 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.311986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.312418 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.313433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:10.313903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.315466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.333172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 409/802] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:10.333980 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.367495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 394/803] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:10.368626 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9111ba-90c7-4aa1-80d5-61a3e46d244a None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.431568 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600420e-b4de-42b1-822b-fccb7e95bd7d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.470849 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.471695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.472402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.472683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.483240 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.483485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9111ba-90c7-4aa1-80d5-61a3e46d244a None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.484733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:10.505766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:10.506903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:10.507914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:10.507914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:10.507914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:10.507914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:10.507914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 410/804] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:10.508228 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.525540 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.526935 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.527220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.527495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.528220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:10.528529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.529923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.541269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 395/805] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:10.541963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.617134 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c21e6-ca67-4f1b-9ae0-496fbfe591de None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.639814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9d31f-3ac1-4fe8-9f5d-bff8cbf00987 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.662267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.663117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.663398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.663766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.674935 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.676378 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.678160 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.678963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.679243 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.679514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.680162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.681705 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:10.693987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:10.694960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:10.695890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:10.695890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:10.695890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:10.695890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:10.696123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:10.697009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:03:10.699431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:10.700246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 411/806] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:10.700331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 396/807] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:10.700331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.701212 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.791005 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78db7e9-84d6-4678-8faf-db54819d741d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.793729 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffbd34-77bd-4d71-811c-a12698c2666e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.822319 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.823872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.824150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.824420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.826134 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6351c-860c-4ac9-8219-c45a972ded18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.826779 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6351c-860c-4ac9-8219-c45a972ded18 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.827056 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6351c-860c-4ac9-8219-c45a972ded18 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.827322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6351c-860c-4ac9-8219-c45a972ded18 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.828027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6351c-860c-4ac9-8219-c45a972ded18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:10.828327 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6351c-860c-4ac9-8219-c45a972ded18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.829721 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6351c-860c-4ac9-8219-c45a972ded18 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.830820 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.832225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.840407 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 412/808] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:10.841612 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b388d1-6b23-49b6-a706-610c81c3b4b9 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.841894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b388d1-6b23-49b6-a706-610c81c3b4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.842165 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b388d1-6b23-49b6-a706-610c81c3b4b9 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.843592 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 413/809] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:10.844102 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b388d1-6b23-49b6-a706-610c81c3b4b9 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:10.847706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:10.848675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:10.850142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 397/810] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:10.854060 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:10.942652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b388d1-6b23-49b6-a706-610c81c3b4b9 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.947545 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bed6d-a54a-4d06-9d7d-dc30dc289c1a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:10.977789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f24ccf-f317-4028-b5cb-5b1e6e0a3e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.978882 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24ccf-f317-4028-b5cb-5b1e6e0a3e8e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.979886 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24ccf-f317-4028-b5cb-5b1e6e0a3e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.980005 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:10.980376 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24ccf-f317-4028-b5cb-5b1e6e0a3e8e None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.980862 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:10.981181 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:10.981354 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f24ccf-f317-4028-b5cb-5b1e6e0a3e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:10.981458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:10.981663 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f24ccf-f317-4028-b5cb-5b1e6e0a3e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.982164 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:10.982474 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:10.983067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f24ccf-f317-4028-b5cb-5b1e6e0a3e8e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.983848 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:10.989157 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 398/811] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:10.989863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.001030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 414/812] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:11.001030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f485bd-4080-4eea-8f1c-d0d377313580 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.001030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f485bd-4080-4eea-8f1c-d0d377313580 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.001030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f485bd-4080-4eea-8f1c-d0d377313580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.023434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-b8f485bd-4080-4eea-8f1c-d0d377313580 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:11.025181 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-b8f485bd-4080-4eea-8f1c-d0d377313580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:11.093897 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73898a8d-03d4-458f-bddc-cba88aabc986 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.098138 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 415/813] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:11.098972 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f485bd-4080-4eea-8f1c-d0d377313580 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.133355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.134149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.134434 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.134711 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.141760 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.143194 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:11.157246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:11.158189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 399/814] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:03:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:11.159138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.196913 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f485bd-4080-4eea-8f1c-d0d377313580 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.230171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.230827 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.231107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.231380 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.232166 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:11.232475 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.233927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.244859 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 416/815] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:11.245744 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.255463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d77f1-f735-4406-aeca-f7dc6e265cb1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.288081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.288734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.289016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.289316 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.290017 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:11.290323 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.291694 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.296535 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 400/816] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:03:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:11.297336 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.340486 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17c3d4-e54d-4529-bfee-050a5770d5aa None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.372036 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.372818 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.373125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.373403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.380109 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.381576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.390381 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f3362-c835-401e-b708-0638329f37a4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:11.396236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:11.397301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:11.398219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 417/817] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:11.398720 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.424600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.425416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.425695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.425968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] PATH_INFO: `/v3/services/network` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.426612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.427959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:11.439704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:11.440792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:11.441728 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 401/818] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:11 2022] GET /identity/v3/services/network => generated 88 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:11.442542 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.497213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dd3c0-aadc-42c3-9201-ee76b983b486 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.529157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.529806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.530087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.530362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.531191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:11.531498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.532920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.537494 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc922386-341d-4e1e-bd01-8a02a326966c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.549121 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 418/819] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:03:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:11.550244 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.570030 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deedc88-d24d-43e1-b644-6bf834dbbcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.570903 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deedc88-d24d-43e1-b644-6bf834dbbcf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.571178 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deedc88-d24d-43e1-b644-6bf834dbbcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.571443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deedc88-d24d-43e1-b644-6bf834dbbcf9 None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/c2567a28c4854d8191aa49947b0a82b8/roles/48c24d23f41249abbb5224c3fa457f9e` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.585496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deedc88-d24d-43e1-b644-6bf834dbbcf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=c2567a28c4854d8191aa49947b0a82b8, role_id=48c24d23f41249abbb5224c3fa457f9e)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.586900 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deedc88-d24d-43e1-b644-6bf834dbbcf9 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.598758 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 402/820] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:03:11 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/c2567a28c4854d8191aa49947b0a82b8/roles/48c24d23f41249abbb5224c3fa457f9e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:03:11.599528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deedc88-d24d-43e1-b644-6bf834dbbcf9 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.652929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75257a0-fbe6-42e1-b39f-191842789a8c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.690701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93ac2-1a62-4135-bc2e-9260fe187246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.691759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93ac2-1a62-4135-bc2e-9260fe187246 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.692271 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93ac2-1a62-4135-bc2e-9260fe187246 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.692749 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93ac2-1a62-4135-bc2e-9260fe187246 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.693713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93ac2-1a62-4135-bc2e-9260fe187246 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.695268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93ac2-1a62-4135-bc2e-9260fe187246 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.718322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 419/821] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:03:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:11.719184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93ac2-1a62-4135-bc2e-9260fe187246 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.723782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deedc88-d24d-43e1-b644-6bf834dbbcf9 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.770717 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.771451 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.771891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.772144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.773072 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:11.773969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.775074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.793484 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 403/822] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:11.794562 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.855987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93ac2-1a62-4135-bc2e-9260fe187246 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.899690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.900699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.901226 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.901649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:11.902538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:11.902994 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:11.904533 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:11.909882 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 420/823] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:11.910873 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:11.924638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd068b7-b5b0-4d93-a55b-16dc0f7afdf7 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:11.967479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c235-21e2-4b18-bbd8-ce82bb74021a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:11.968137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c235-21e2-4b18-bbd8-ce82bb74021a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:11.968418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c235-21e2-4b18-bbd8-ce82bb74021a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:11.968696 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c235-21e2-4b18-bbd8-ce82bb74021a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:12.025255 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd43bd-87ae-4201-a226-e78d217b91e6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:12.051381 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c235-21e2-4b18-bbd8-ce82bb74021a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:12.053339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c235-21e2-4b18-bbd8-ce82bb74021a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:12.065245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394ae98-b132-487b-a3aa-0615cdd22b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:12.066468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394ae98-b132-487b-a3aa-0615cdd22b16 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:12.066753 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394ae98-b132-487b-a3aa-0615cdd22b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:12.067031 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394ae98-b132-487b-a3aa-0615cdd22b16 None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/14a8942a978b4d92be89376cd31a1240/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:12.069541 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394ae98-b132-487b-a3aa-0615cdd22b16 None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=14a8942a978b4d92be89376cd31a1240, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:12.071627 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c394ae98-b132-487b-a3aa-0615cdd22b16 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:12.089777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 421/824] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:03:11 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/14a8942a978b4d92be89376cd31a1240/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:03:12.090721 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394ae98-b132-487b-a3aa-0615cdd22b16 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:12.180557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 404/825] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:03:12.190198 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c235-21e2-4b18-bbd8-ce82bb74021a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:12.245649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394ae98-b132-487b-a3aa-0615cdd22b16 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:12.311578 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c235-21e2-4b18-bbd8-ce82bb74021a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:12.326033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:12.326981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:12.327406 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:12.327835 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:12.328696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] RBAC: Adding query filter params (type=network) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:12.330753 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:12.332296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:12.339220 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 422/826] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:12.346814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:12.366957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:12.367762 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:12.368042 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:12.368315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:12.381646 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:12.383622 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:12.408253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:12.409420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:12.410526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:12.410526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:12.410526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:12.410526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:12.410526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 405/827] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:12.417959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:12.526009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fa396-bed3-451d-aa08-d8b2ed187aaf None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:12.574803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56151c-fa13-46b4-8af9-6cb078af9ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:12.577378 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56151c-fa13-46b4-8af9-6cb078af9ea2 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:12.577894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56151c-fa13-46b4-8af9-6cb078af9ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:12.578602 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56151c-fa13-46b4-8af9-6cb078af9ea2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:12.579664 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56151c-fa13-46b4-8af9-6cb078af9ea2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:12.585111 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56151c-fa13-46b4-8af9-6cb078af9ea2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:12.613892 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 423/828] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:03:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:03:12.655393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694030c7-70ca-46b2-a3fd-5c72e05ff786 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:12.723854 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:12.724891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:12.725564 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:12.727082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:12.728359 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:12.729688 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:12.731532 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:12.751326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 406/829] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:12.759487 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56151c-fa13-46b4-8af9-6cb078af9ea2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:12.917803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56151c-fa13-46b4-8af9-6cb078af9ea2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:12.966098 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:12.967239 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:12.967714 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:12.968202 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:12.975363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:12.977207 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:12.994334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:12.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:12.996408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:12.996408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:12.996408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:12.996408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:12.996408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:12.996408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:12.996408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:12.996408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:12.998986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 424/830] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:13.004774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:13.137233 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ad971-81dc-4de6-9577-6336320a1cc0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:13.177955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10bdee-a673-411a-9d08-187314ab7436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:13.178940 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10bdee-a673-411a-9d08-187314ab7436 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:13.179415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10bdee-a673-411a-9d08-187314ab7436 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:13.179879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10bdee-a673-411a-9d08-187314ab7436 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:13.180879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10bdee-a673-411a-9d08-187314ab7436 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:13.181938 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10bdee-a673-411a-9d08-187314ab7436 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:13.184173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10bdee-a673-411a-9d08-187314ab7436 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:13.197486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 407/831] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:13.206265 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:13.307153 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b33bb7-0ac4-4c3b-a3af-eae046ef809f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:13.341170 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c62e8c-2fd9-4f9e-897d-62a0dca2a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:13.341823 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c62e8c-2fd9-4f9e-897d-62a0dca2a953 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:13.342107 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c62e8c-2fd9-4f9e-897d-62a0dca2a953 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:13.342385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c62e8c-2fd9-4f9e-897d-62a0dca2a953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:13.343222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c62e8c-2fd9-4f9e-897d-62a0dca2a953 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=512c115af57c463eb1fbe636667306dd) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:13.343537 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c62e8c-2fd9-4f9e-897d-62a0dca2a953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:13.345018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c62e8c-2fd9-4f9e-897d-62a0dca2a953 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:13.357190 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 425/832] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:13 2022] GET /identity/v3/role_assignments?user.id=512c115af57c463eb1fbe636667306dd&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:14.063533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c579ba-94d3-4c7c-8496-04f3e36e27e1 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.063943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c579ba-94d3-4c7c-8496-04f3e36e27e1 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.064132 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c579ba-94d3-4c7c-8496-04f3e36e27e1 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.065723 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 408/833] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:14.071221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8a836-35ba-4cf9-b38e-857948f481d3 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.071505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8a836-35ba-4cf9-b38e-857948f481d3 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.071781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8a836-35ba-4cf9-b38e-857948f481d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.089230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-c0e8a836-35ba-4cf9-b38e-857948f481d3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:14.090417 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-c0e8a836-35ba-4cf9-b38e-857948f481d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:14.161485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 426/834] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:14.184837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c579ba-94d3-4c7c-8496-04f3e36e27e1 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:14.288257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c579ba-94d3-4c7c-8496-04f3e36e27e1 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:14.321859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13ab92-fdd8-46b1-80bf-3c1f1aa0342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:14.322568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13ab92-fdd8-46b1-80bf-3c1f1aa0342d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.323396 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13ab92-fdd8-46b1-80bf-3c1f1aa0342d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.323682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13ab92-fdd8-46b1-80bf-3c1f1aa0342d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.333839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13ab92-fdd8-46b1-80bf-3c1f1aa0342d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-8e13ab92-fdd8-46b1-80bf-3c1f1aa0342d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:14.341505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:14.342432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:14.343278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 409/835] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:03:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:14.348774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8a836-35ba-4cf9-b38e-857948f481d3 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:14.358574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8791-76a6-4ab8-96a3-c6c861e02481 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.358862 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8791-76a6-4ab8-96a3-c6c861e02481 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.359143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8791-76a6-4ab8-96a3-c6c861e02481 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.360472 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 410/836] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:14.365372 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d1fe-c01e-4cc6-9b0c-7f78ad1be112 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.365632 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d1fe-c01e-4cc6-9b0c-7f78ad1be112 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.365913 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004d1fe-c01e-4cc6-9b0c-7f78ad1be112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.383000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-f004d1fe-c01e-4cc6-9b0c-7f78ad1be112 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:14.384174 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-f004d1fe-c01e-4cc6-9b0c-7f78ad1be112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:14.455356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 411/837] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:14.461982 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8a836-35ba-4cf9-b38e-857948f481d3 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:14.471537 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004d1fe-c01e-4cc6-9b0c-7f78ad1be112 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:14.505488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:14.506167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.506446 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.506719 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.508514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:14.508839 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:14.510333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:14.522276 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 427/838] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:03:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:14.527707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:14.579229 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004d1fe-c01e-4cc6-9b0c-7f78ad1be112 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:14.613830 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:14.614790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.615177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.615522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.616243 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:14.617708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:14.628245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae69cd-0dae-4fcb-aa99-ee9a1376034b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:14.629867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:14.631784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:14.633826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:14.641150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 412/839] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:14.641150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:14.663288 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c917b-4961-461e-b4ef-6f48b2bfc769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:14.664080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c917b-4961-461e-b4ef-6f48b2bfc769 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.664357 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c917b-4961-461e-b4ef-6f48b2bfc769 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.664629 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c917b-4961-461e-b4ef-6f48b2bfc769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.675679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c917b-4961-461e-b4ef-6f48b2bfc769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:14.677179 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c917b-4961-461e-b4ef-6f48b2bfc769 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-962c917b-4961-461e-b4ef-6f48b2bfc769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:14.694505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:14.695624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:14.696730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:14.696730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:14.696730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:14.696730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:14.697196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 428/840] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:14.699112 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3290-167c-4d00-a28a-7ed5b3812362 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.699493 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3290-167c-4d00-a28a-7ed5b3812362 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.699863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e3290-167c-4d00-a28a-7ed5b3812362 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.701839 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 429/841] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:14.705872 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e3290-167c-4d00-a28a-7ed5b3812362 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:14.750066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e339c3f-7176-455d-b358-fbc387e66560 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:14.782921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04794a1-1229-49bf-a228-d00743a389bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:14.783575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04794a1-1229-49bf-a228-d00743a389bf None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.783854 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04794a1-1229-49bf-a228-d00743a389bf None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.784127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04794a1-1229-49bf-a228-d00743a389bf None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.784879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04794a1-1229-49bf-a228-d00743a389bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:14.785218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04794a1-1229-49bf-a228-d00743a389bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:14.786623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04794a1-1229-49bf-a228-d00743a389bf None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:14.796022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 413/842] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:14.796022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae423-8b2f-4882-bccf-e4a85556d68f None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.796022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae423-8b2f-4882-bccf-e4a85556d68f None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.796022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae423-8b2f-4882-bccf-e4a85556d68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.804756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e3290-167c-4d00-a28a-7ed5b3812362 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:14.812349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-52cae423-8b2f-4882-bccf-e4a85556d68f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:14.813203 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-52cae423-8b2f-4882-bccf-e4a85556d68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:14.844314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:14.844899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:14.845229 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:14.845505 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:14.846219 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:14.846520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:14.847867 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:14.864125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 430/843] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:14.864125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:14.885017 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 414/844] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:14.886139 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cae423-8b2f-4882-bccf-e4a85556d68f None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:14.971711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6dd9-f9e0-4fda-9e9c-c474f8b6c151 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:14.991719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cae423-8b2f-4882-bccf-e4a85556d68f None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.006949 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.007892 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.008299 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.008699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.009690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:15.010233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:15.011723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:15.022754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 431/845] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:15.022754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.033122 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.033966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.034246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.035238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.046227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:15.048291 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:15.069497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:15.070518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:15.071473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:15.076738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 415/846] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:03:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:15.077364 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.152296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12222fd8-2dbd-46ff-8476-b2582833de2a None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.188561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ade3-5002-47eb-9005-ffa27b45ed0b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.191040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6566b21-c206-444a-ad8a-643b94b82c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.192086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6566b21-c206-444a-ad8a-643b94b82c94 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.192570 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6566b21-c206-444a-ad8a-643b94b82c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.193074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6566b21-c206-444a-ad8a-643b94b82c94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.206226 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6566b21-c206-444a-ad8a-643b94b82c94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a6566b21-c206-444a-ad8a-643b94b82c94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:15.214413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:15.215499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:15.216498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:15.223440 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 432/847] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:15.224363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6566b21-c206-444a-ad8a-643b94b82c94 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.224872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.225594 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.226298 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.226298 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.226932 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:15.227240 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:15.228654 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:15.250981 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 416/848] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:15.257101 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.341884 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6566b21-c206-444a-ad8a-643b94b82c94 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.372802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad5076-d500-40de-a628-01907d8e55a3 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.382714 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.383633 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.383919 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.384191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.385347 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:15.386160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:15.387800 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:15.409074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 433/849] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:15.412349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.425753 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206b74b-5bed-4444-80f5-0bef1c3c2b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.426753 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206b74b-5bed-4444-80f5-0bef1c3c2b0a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.427237 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206b74b-5bed-4444-80f5-0bef1c3c2b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.427788 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206b74b-5bed-4444-80f5-0bef1c3c2b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.514556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206b74b-5bed-4444-80f5-0bef1c3c2b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:15.516320 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206b74b-5bed-4444-80f5-0bef1c3c2b0a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:15.559330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3a38e-6489-4222-83dd-5516741e70e2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.612797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.613931 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.614544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.615013 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.627419 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:15.629651 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:15.647278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 417/850] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:15.649735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:15.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:15.652159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:15.652159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:15.652159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:15.652159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:15.652785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206b74b-5bed-4444-80f5-0bef1c3c2b0a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.658566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 434/851] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:15.659080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.764455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c34c-ada9-4062-9aea-089cd072bb6b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.773522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206b74b-5bed-4444-80f5-0bef1c3c2b0a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.799133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be65001-6029-48f8-b6cc-acb22bb7ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.799793 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be65001-6029-48f8-b6cc-acb22bb7ec40 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.800073 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be65001-6029-48f8-b6cc-acb22bb7ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.800569 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be65001-6029-48f8-b6cc-acb22bb7ec40 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.801089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be65001-6029-48f8-b6cc-acb22bb7ec40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:15.801399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be65001-6029-48f8-b6cc-acb22bb7ec40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:15.802763 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be65001-6029-48f8-b6cc-acb22bb7ec40 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:15.808432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.809248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.809530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.809805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.817500 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 435/852] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:15.817500 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235460-2107-4e8c-839e-2c06c39c0c31 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.817500 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235460-2107-4e8c-839e-2c06c39c0c31 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.817500 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff235460-2107-4e8c-839e-2c06c39c0c31 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.818096 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 436/853] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:15.821662 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:15.823863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:15.824306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f8147-7183-445a-b0c7-1a1e78d4a4ce None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.824593 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f8147-7183-445a-b0c7-1a1e78d4a4ce None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.824869 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f8147-7183-445a-b0c7-1a1e78d4a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:15.841361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-792f8147-7183-445a-b0c7-1a1e78d4a4ce None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:15.842049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:15.842917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:15.844458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:15.844458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:15.844458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:15.844458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:15.844685 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-792f8147-7183-445a-b0c7-1a1e78d4a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:15.844756 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 418/854] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:15.844756 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.911648 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 437/855] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:15.912399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f8147-7183-445a-b0c7-1a1e78d4a4ce None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:15.959734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2965e4-1f95-4893-b770-ef81b80207b6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:15.995311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:15.996386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:15.996386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:15.996658 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.004012 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.005459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:16.019492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:16.020442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:03:16.021427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:16.022178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f8147-7183-445a-b0c7-1a1e78d4a4ce None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.025362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 419/856] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:16.026475 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.055700 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.056361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.056642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.056918 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.057666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:16.057972 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.059334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.070371 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 438/857] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:16.071408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.124449 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c6c19-3def-437a-ac17-fd87d64a10ef None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.158846 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.159804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.160084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.160360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.161013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.162443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.169862 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51db9-d153-434e-8084-f42eb6893dc9 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:16.174879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:16.175801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:16.176639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:16.180892 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 420/858] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:16.181705 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.205870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.206557 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.206830 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.207099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.207952 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:16.208255 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.209785 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.231608 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 439/859] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:16.231608 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.281370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675dc4d8-c174-4127-993c-e88e60770dbd None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.314101 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.315028 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.315311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.315578 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.322166 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.323612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.329498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe8084-73a9-4ba6-bd73-6c00c164094f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:16.337465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:16.338584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 421/860] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:16.339498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.368053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.368926 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.369367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.370523 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.371365 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:16.371808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.373334 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.380987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 440/861] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:16.381713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.439601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659bacb-7065-45a6-b420-0849dfb7ca00 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.471977 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324af3f-8a74-4171-8974-494121cbbca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.472618 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324af3f-8a74-4171-8974-494121cbbca1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.472896 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324af3f-8a74-4171-8974-494121cbbca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.473200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324af3f-8a74-4171-8974-494121cbbca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.477779 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b7dba-fd77-4536-a913-c0d5b8a03aca None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.509966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.511145 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.511426 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.511698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.512482 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:16.512789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.514238 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.530341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 441/862] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:16.530341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.545359 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324af3f-8a74-4171-8974-494121cbbca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.546797 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324af3f-8a74-4171-8974-494121cbbca1 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.630418 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381a3d3-8ed4-4522-ad88-58c01dbe859f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.646840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 422/863] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:03:16.646840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324af3f-8a74-4171-8974-494121cbbca1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.668799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.669705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.670147 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.670564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.671468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:16.671914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.673471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.678431 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 442/864] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:16.679421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:16.766754 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324af3f-8a74-4171-8974-494121cbbca1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.805920 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.806974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.808029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.808480 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.809566 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=c2567a28c4854d8191aa49947b0a82b8) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:16.810119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.811758 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.818641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 423/865] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/role_assignments?user.id=c2567a28c4854d8191aa49947b0a82b8&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:16.829662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44271c0d-d5cf-4ff4-8195-d41eb2d06113 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:16.868480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:16.870396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:16.870860 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:16.871306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:16.883596 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:16.885108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:16.906634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:16.907769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:16.908878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:16.908878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:16.908878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:16.908878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:16.910537 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 443/866] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:16.916998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:17.037912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e374d2-8514-475e-8081-a1359ddfec3f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:17.072213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:17.073810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:17.074328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:17.074819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:17.075745 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:17.076251 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:17.077832 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:17.090449 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 424/867] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:17.095746 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:17.206488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e8ced-c047-4df6-ae79-0a5972db65db None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:17.241446 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86a9db-dd6f-4163-959e-15cd6a8509dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:17.242635 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a9db-dd6f-4163-959e-15cd6a8509dc None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:17.243123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a9db-dd6f-4163-959e-15cd6a8509dc None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:17.243597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a9db-dd6f-4163-959e-15cd6a8509dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:17.251553 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86a9db-dd6f-4163-959e-15cd6a8509dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:17.253187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86a9db-dd6f-4163-959e-15cd6a8509dc None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-fb86a9db-dd6f-4163-959e-15cd6a8509dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:17.267973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:17.269103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:17.270192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:17.270192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:17.270192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:17.270192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:17.270192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:17.270192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:17.270192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:17.270192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:17.272012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 444/868] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:17.277883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:17.340317 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992b479-155f-4576-b294-2fa9c57c4485 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:17.340527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992b479-155f-4576-b294-2fa9c57c4485 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:17.340804 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992b479-155f-4576-b294-2fa9c57c4485 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:17.342474 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 445/869] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:17.347388 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3d77-aaf5-4016-85ba-1573882846db None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:17.347674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3d77-aaf5-4016-85ba-1573882846db None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:17.347957 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3d77-aaf5-4016-85ba-1573882846db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:17.365170 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-2f5d3d77-aaf5-4016-85ba-1573882846db None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:17.366365 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-2f5d3d77-aaf5-4016-85ba-1573882846db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:17.380904 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42d7d4-24d2-41a4-9024-777ff313775c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:17.418080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:17.419018 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:17.419492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:17.419930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:17.420900 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:17.421421 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:17.423070 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:17.433845 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 446/870] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:17.436816 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 425/871] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:17.445078 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d3d77-aaf5-4016-85ba-1573882846db None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:17.458083 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:17.556218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b60298-caf7-478d-8f0f-ae399716815f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:17.566642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d3d77-aaf5-4016-85ba-1573882846db None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:17.597256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f697fe-61f9-4cf1-825d-15300d942802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:17.597869 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f697fe-61f9-4cf1-825d-15300d942802 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:17.598132 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f697fe-61f9-4cf1-825d-15300d942802 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:17.598414 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f697fe-61f9-4cf1-825d-15300d942802 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:17.599387 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f697fe-61f9-4cf1-825d-15300d942802 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:17.600848 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f697fe-61f9-4cf1-825d-15300d942802 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:17.604057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51b027-355a-4a95-b4f0-2e51259991bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:17.604959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51b027-355a-4a95-b4f0-2e51259991bd None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:17.605474 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51b027-355a-4a95-b4f0-2e51259991bd None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:17.606017 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51b027-355a-4a95-b4f0-2e51259991bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:17.606941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51b027-355a-4a95-b4f0-2e51259991bd None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=14a8942a978b4d92be89376cd31a1240) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:17.607328 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51b027-355a-4a95-b4f0-2e51259991bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:17.608918 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51b027-355a-4a95-b4f0-2e51259991bd None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:17.621102 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 447/872] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:17 2022] GET /identity/v3/role_assignments?user.id=14a8942a978b4d92be89376cd31a1240&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:17.625752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 426/873] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:03:17 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:03:19.020461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7be5e-adba-4c17-a6f0-acd854295700 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.021129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7be5e-adba-4c17-a6f0-acd854295700 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.021129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7be5e-adba-4c17-a6f0-acd854295700 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.022657 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 448/874] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:19.028895 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c2143-3f8f-4099-8d4c-b9972baf5a32 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.030106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c2143-3f8f-4099-8d4c-b9972baf5a32 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.030864 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c2143-3f8f-4099-8d4c-b9972baf5a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.050508 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-ee0c2143-3f8f-4099-8d4c-b9972baf5a32 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:19.052083 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-ee0c2143-3f8f-4099-8d4c-b9972baf5a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:19.141147 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 427/875] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:19.157994 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7be5e-adba-4c17-a6f0-acd854295700 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:19.265008 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7be5e-adba-4c17-a6f0-acd854295700 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:19.279285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873080-5413-466c-932e-562100c5724d None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.279550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873080-5413-466c-932e-562100c5724d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.279770 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1873080-5413-466c-932e-562100c5724d None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.281278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 428/876] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:19.286121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afafbb7-c09d-4dcf-a63e-46204b6282a5 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.286403 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afafbb7-c09d-4dcf-a63e-46204b6282a5 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.286680 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afafbb7-c09d-4dcf-a63e-46204b6282a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.305782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-8afafbb7-c09d-4dcf-a63e-46204b6282a5 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:19.307581 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-8afafbb7-c09d-4dcf-a63e-46204b6282a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:19.308212 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6565af-301b-40b2-bf12-102c918fc077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:19.309124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6565af-301b-40b2-bf12-102c918fc077 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.309622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6565af-301b-40b2-bf12-102c918fc077 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.310110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6565af-301b-40b2-bf12-102c918fc077 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.311050 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6565af-301b-40b2-bf12-102c918fc077 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:19.312630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6565af-301b-40b2-bf12-102c918fc077 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:19.330841 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 449/877] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:03:19 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:19.393197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 429/878] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:19.412611 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6565af-301b-40b2-bf12-102c918fc077 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:19.528369 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6565af-301b-40b2-bf12-102c918fc077 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:19.565223 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c15b0-e1c3-4708-b8fe-7001afea8ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:19.566018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c15b0-e1c3-4708-b8fe-7001afea8ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.566296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c15b0-e1c3-4708-b8fe-7001afea8ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.566561 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c15b0-e1c3-4708-b8fe-7001afea8ae4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.577483 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c15b0-e1c3-4708-b8fe-7001afea8ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-de8c15b0-e1c3-4708-b8fe-7001afea8ae4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:19.585857 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:19.586922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:19.587840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:19.589152 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 450/879] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:03:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:19.593594 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afafbb7-c09d-4dcf-a63e-46204b6282a5 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:19.693030 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afafbb7-c09d-4dcf-a63e-46204b6282a5 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:19.726939 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:19.727708 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.727991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.728268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.729185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:19.729491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:19.730847 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:19.742755 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 430/880] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:03:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:19.746176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c15b0-e1c3-4708-b8fe-7001afea8ae4 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:19.852845 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c15b0-e1c3-4708-b8fe-7001afea8ae4 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:19.890818 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c437925-3d39-40ba-a40b-ac799afddbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:19.891626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437925-3d39-40ba-a40b-ac799afddbdb None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.891900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437925-3d39-40ba-a40b-ac799afddbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.892172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437925-3d39-40ba-a40b-ac799afddbdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.905563 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c437925-3d39-40ba-a40b-ac799afddbdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:19.907328 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c437925-3d39-40ba-a40b-ac799afddbdb None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-4c437925-3d39-40ba-a40b-ac799afddbdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:19.925131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:19.927256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:19.929509 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:19.929509 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:19.929509 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:19.929509 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:19.932877 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 451/881] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:19.933103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:19.948670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66133d-917e-49bd-bec9-1172f9ef908f None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.948670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66133d-917e-49bd-bec9-1172f9ef908f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.948670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66133d-917e-49bd-bec9-1172f9ef908f None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.948670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 452/882] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:19.953529 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa70d1-4dd8-485d-b4b2-44881c90d65f None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:19.953810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa70d1-4dd8-485d-b4b2-44881c90d65f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:19.954084 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa70d1-4dd8-485d-b4b2-44881c90d65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:19.975215 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-beaa70d1-4dd8-485d-b4b2-44881c90d65f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:19.976821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-beaa70d1-4dd8-485d-b4b2-44881c90d65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:20.043303 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1e130-1f8e-4534-8875-c00be7be1360 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.059685 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 453/883] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:20.064363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed0eba-e7ad-4b47-8a89-5f5f163686f3 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.064642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed0eba-e7ad-4b47-8a89-5f5f163686f3 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.064914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed0eba-e7ad-4b47-8a89-5f5f163686f3 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.066511 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 454/884] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:20.071393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6233d8a-e7e1-4525-b62d-7fa3c334ef14 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.071678 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6233d8a-e7e1-4525-b62d-7fa3c334ef14 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.071951 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6233d8a-e7e1-4525-b62d-7fa3c334ef14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.078272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.078929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.079358 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.079630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.080374 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:20.080677 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.082056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.089295 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-e6233d8a-e7e1-4525-b62d-7fa3c334ef14 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:20.090652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-e6233d8a-e7e1-4525-b62d-7fa3c334ef14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:20.095546 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 431/885] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:20.099288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.159760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 455/886] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:20.160927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6233d8a-e7e1-4525-b62d-7fa3c334ef14 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.207778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7516038-6fd2-4e58-8a8c-3d21063c7191 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.240169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.240952 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.241255 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.241519 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.242162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.243504 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:20.256338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:20.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:20.258286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 432/887] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:20.259107 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.264601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6233d8a-e7e1-4525-b62d-7fa3c334ef14 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.298306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.299130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.299412 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.299686 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.306697 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.308687 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:20.323856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:20.325020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:20.326027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:20.326591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 456/888] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:20.327032 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.362392 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309ef18-6b51-40e7-a7f3-2013b8a1d583 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.395278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.396339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.396621 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.396894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.397576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.398936 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:20.412390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:20.413476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:20.414321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:20.420956 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 433/889] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:20.420956 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.424323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445884b-980f-455d-b5e2-6fc798557992 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.459855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.460514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.460798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.461099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.461806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:20.462114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.463508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.472317 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 457/890] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:20.472317 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.532040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497df31f-991b-4c67-9cea-e8c14951d7b0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.574918 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee5c57-e200-45b7-a1f8-bbdce352c30f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.575411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.576249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.576527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.576794 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.577665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:20.577972 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.579374 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.600941 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 434/891] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:20.600941 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.612821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.613504 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.613782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.614076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.614746 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:20.615050 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.616405 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.626875 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 458/892] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:20.627713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.692450 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a8aca-ee78-4999-85bf-a69a5ae90fef None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.722382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaca04c-6180-4be3-890a-9dcb623b6c65 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.723885 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.724539 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.724814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.725097 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.725795 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:20.726097 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.727443 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.732699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 435/893] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:20.733485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.761814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6d4f9-b619-4108-84ab-f897915b4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.762729 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6d4f9-b619-4108-84ab-f897915b4d62 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.763184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6d4f9-b619-4108-84ab-f897915b4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.763641 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6d4f9-b619-4108-84ab-f897915b4d62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.827769 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193410-7c85-45b6-8df6-67041d7845f2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:20.834294 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6d4f9-b619-4108-84ab-f897915b4d62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.835765 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6d4f9-b619-4108-84ab-f897915b4d62 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.864831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:20.865521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:20.865798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:20.866229 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:20.866796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:20.867065 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:20.868413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:20.881384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 436/894] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:20.881384 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:20.941555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 459/895] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:03:20.947092 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6d4f9-b619-4108-84ab-f897915b4d62 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:21.016967 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a51ad-431a-446d-85cb-0a29070df93a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:21.055390 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6d4f9-b619-4108-84ab-f897915b4d62 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:21.058261 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:21.059534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.060024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.060529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.061568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] RBAC: Adding query filter params (interface=public, service_id=d40ee01252eb45d3b200dc1b6f006d1c, region_id=RegionOne) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:21.062105 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:21.063719 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:21.072430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 437/896] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/endpoints?service_id=d40ee01252eb45d3b200dc1b6f006d1c&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:21.078113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:21.098330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:21.099120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.099402 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.099675 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.111465 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:21.115208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:21.138043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:21.139113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:21.140167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:21.140167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:21.140167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:21.140167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:21.140167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 460/897] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:21.145911 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:21.186284 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85266937-1fff-41f9-bd08-8227d09e4814 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:21.220814 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5147f3d-db3d-42e2-b966-a26566c1fa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:21.221137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147f3d-db3d-42e2-b966-a26566c1fa00 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.221421 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147f3d-db3d-42e2-b966-a26566c1fa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.221696 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5147f3d-db3d-42e2-b966-a26566c1fa00 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.222381 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5147f3d-db3d-42e2-b966-a26566c1fa00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:21.223811 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5147f3d-db3d-42e2-b966-a26566c1fa00 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:21.233103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 438/898] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:03:21 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:03:21.258048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f39b93-893a-4303-8597-8b19711ab6db None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:21.297386 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:21.298326 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.298787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.299233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.300154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:21.300622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:21.302242 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:21.316088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 461/899] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:21.322001 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5147f3d-db3d-42e2-b966-a26566c1fa00 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:21.426155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5147f3d-db3d-42e2-b966-a26566c1fa00 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:21.465367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:21.466156 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.466429 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.466695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.473958 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:21.475330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:21.490478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:21.491466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:21.492405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 439/900] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:21.500306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:21.604974 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fee1e-12dc-4188-a3bb-9ccdc8b8bae8 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:21.639527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537806-8b5c-48fe-8a01-5011632d5636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:21.640606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537806-8b5c-48fe-8a01-5011632d5636 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.641115 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537806-8b5c-48fe-8a01-5011632d5636 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.641600 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537806-8b5c-48fe-8a01-5011632d5636 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.642589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00537806-8b5c-48fe-8a01-5011632d5636 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:21.643089 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00537806-8b5c-48fe-8a01-5011632d5636 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:21.644713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00537806-8b5c-48fe-8a01-5011632d5636 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:21.657589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 462/901] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:21.663798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:21.683584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1bd4c-80e0-4e63-ad20-a2ac569ff9cf None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.683912 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1bd4c-80e0-4e63-ad20-a2ac569ff9cf None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.684160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1bd4c-80e0-4e63-ad20-a2ac569ff9cf None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.685797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 463/902] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:21.690730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8906f-ac70-4c9f-b178-08602292466c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.691015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8906f-ac70-4c9f-b178-08602292466c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.691286 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8906f-ac70-4c9f-b178-08602292466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.708264 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-c5f8906f-ac70-4c9f-b178-08602292466c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:21.709487 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-c5f8906f-ac70-4c9f-b178-08602292466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:21.781127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1f986-9771-4bf1-9699-3fc990521d02 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:21.781322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 464/903] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:21.799499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8906f-ac70-4c9f-b178-08602292466c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:21.818851 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:21.819838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.820325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.821224 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.822276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] RBAC: Adding query filter params (role.id=fa08f0adc60e48b192f5e9adecef99a5, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=c2567a28c4854d8191aa49947b0a82b8) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:21.823633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:21.825739 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:21.833431 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 440/904] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:21 2022] GET /identity/v3/role_assignments?user.id=c2567a28c4854d8191aa49947b0a82b8&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=fa08f0adc60e48b192f5e9adecef99a5 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:21.908610 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8906f-ac70-4c9f-b178-08602292466c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:21.944304 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:21.945211 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:21.945541 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:21.945821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:21.946479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:21.948452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:21.965948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:21.966979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:21.967808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:21.970915 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 465/905] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:03:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:21.976910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:22.085987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cda4d-8535-49dd-96ff-466d1a43c42c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:22.119964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:22.120674 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:22.120959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:22.121263 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:22.122164 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:22.122474 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:22.123844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:22.129222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 441/906] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:22.135700 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:22.239468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5eea5-943f-4157-886a-39ec09870b6b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:22.273137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:22.274154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:22.274631 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:22.275117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:22.276066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:22.276577 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:22.278167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:22.283611 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 466/907] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:22.289281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:22.389561 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843840b2-3164-4e86-a2a0-c875ce74a22b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:22.423046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02410f9-570c-469d-b8c3-be8c6cc744e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:22.423757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02410f9-570c-469d-b8c3-be8c6cc744e7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:22.424043 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02410f9-570c-469d-b8c3-be8c6cc744e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:22.424320 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02410f9-570c-469d-b8c3-be8c6cc744e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:22.425133 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02410f9-570c-469d-b8c3-be8c6cc744e7 None admin] RBAC: Adding query filter params (interface=public, service_id=5ecd4530f2cc44669964758ee7b7a6c5, region_id=RegionOne) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:22.425446 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02410f9-570c-469d-b8c3-be8c6cc744e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:22.426802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02410f9-570c-469d-b8c3-be8c6cc744e7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:22.432882 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 442/908] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:03:22 2022] GET /identity/v3/endpoints?service_id=5ecd4530f2cc44669964758ee7b7a6c5&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:22.438270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:22.544002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfb363-f59c-4926-8c9c-93014a801356 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:22.578182 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e46fb-30d1-48ac-99a4-e45d950a7a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:22.579202 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e46fb-30d1-48ac-99a4-e45d950a7a32 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:22.579679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e46fb-30d1-48ac-99a4-e45d950a7a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:22.580163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e46fb-30d1-48ac-99a4-e45d950a7a32 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:22.581055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e46fb-30d1-48ac-99a4-e45d950a7a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:22.582627 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e46fb-30d1-48ac-99a4-e45d950a7a32 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:22.589442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 467/909] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:03:22 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:03:23.195841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ad8e4-4921-4d70-8f44-92c00ce7cbf0 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:23.196747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ad8e4-4921-4d70-8f44-92c00ce7cbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:23.197213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ad8e4-4921-4d70-8f44-92c00ce7cbf0 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:23.199029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 443/910] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:23.205257 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453012e-c2a1-42c5-8ffe-4df178a629d2 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:23.206346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453012e-c2a1-42c5-8ffe-4df178a629d2 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:23.206769 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453012e-c2a1-42c5-8ffe-4df178a629d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:23.224738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-6453012e-c2a1-42c5-8ffe-4df178a629d2 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:23.226130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-6453012e-c2a1-42c5-8ffe-4df178a629d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:23.298247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 468/911] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:23.320690 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ad8e4-4921-4d70-8f44-92c00ce7cbf0 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:23.429643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ad8e4-4921-4d70-8f44-92c00ce7cbf0 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:23.463915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a77554-5314-4e26-b23c-064da6cc8513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:23.464573 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a77554-5314-4e26-b23c-064da6cc8513 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:23.464851 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a77554-5314-4e26-b23c-064da6cc8513 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:23.465190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a77554-5314-4e26-b23c-064da6cc8513 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:23.465929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a77554-5314-4e26-b23c-064da6cc8513 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:23.467282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a77554-5314-4e26-b23c-064da6cc8513 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:23.488468 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 444/912] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:03:23 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:23.593580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81996ee8-fc27-41e7-9297-9dcbbfaa53f8 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:23.594226 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81996ee8-fc27-41e7-9297-9dcbbfaa53f8 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:23.594706 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81996ee8-fc27-41e7-9297-9dcbbfaa53f8 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:23.596607 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 469/913] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:23.602502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4aad8-b2be-4aaa-8087-3b5a49a04a35 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:23.603112 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4aad8-b2be-4aaa-8087-3b5a49a04a35 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:23.603603 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4aad8-b2be-4aaa-8087-3b5a49a04a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:23.622060 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-77a4aad8-b2be-4aaa-8087-3b5a49a04a35 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:23.623638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-77a4aad8-b2be-4aaa-8087-3b5a49a04a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:23.702735 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 445/914] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:23.725881 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81996ee8-fc27-41e7-9297-9dcbbfaa53f8 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:23.832839 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81996ee8-fc27-41e7-9297-9dcbbfaa53f8 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:23.868027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:23.869267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:23.870287 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:23.870787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:23.871661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:23.873337 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:23.887039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:23.888064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:03:23.888944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:23.891884 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 470/915] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:23.898214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4aad8-b2be-4aaa-8087-3b5a49a04a35 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:24.008084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4aad8-b2be-4aaa-8087-3b5a49a04a35 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:24.043863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac8755-ecb2-46cc-afa4-78036e42fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:24.044901 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8755-ecb2-46cc-afa4-78036e42fc75 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.045442 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8755-ecb2-46cc-afa4-78036e42fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.045910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8755-ecb2-46cc-afa4-78036e42fc75 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.046829 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac8755-ecb2-46cc-afa4-78036e42fc75 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:24.047356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac8755-ecb2-46cc-afa4-78036e42fc75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:24.048895 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac8755-ecb2-46cc-afa4-78036e42fc75 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:24.054057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 446/916] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:24.060347 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:24.168832 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca129e-243d-4faf-9ec8-72a7f8f712ef None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.169793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca129e-243d-4faf-9ec8-72a7f8f712ef None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.170530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca129e-243d-4faf-9ec8-72a7f8f712ef None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.172768 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 447/917] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:24.177915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fb435-78ae-40c0-ab4d-9f4f4c6834b5 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.178448 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fb435-78ae-40c0-ab4d-9f4f4c6834b5 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.178554 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d98a0e-d513-4a04-b654-408e54248b5b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:24.179045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fb435-78ae-40c0-ab4d-9f4f4c6834b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.196672 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-325fb435-78ae-40c0-ab4d-9f4f4c6834b5 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:24.198456 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-325fb435-78ae-40c0-ab4d-9f4f4c6834b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:24.218009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540de0-2411-4020-88eb-c91343135feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:24.219346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540de0-2411-4020-88eb-c91343135feb None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.219824 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540de0-2411-4020-88eb-c91343135feb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.220267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540de0-2411-4020-88eb-c91343135feb None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.221243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28540de0-2411-4020-88eb-c91343135feb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:24.221720 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28540de0-2411-4020-88eb-c91343135feb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:24.227251 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28540de0-2411-4020-88eb-c91343135feb None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:24.238015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 471/918] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:03:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:24.239100 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540de0-2411-4020-88eb-c91343135feb None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:24.273293 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 448/919] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:24.290495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fb435-78ae-40c0-ab4d-9f4f4c6834b5 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:24.365084 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540de0-2411-4020-88eb-c91343135feb None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:24.399412 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef0205-fb52-47a6-8a49-4e85306a521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:24.400068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef0205-fb52-47a6-8a49-4e85306a521b None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.400349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef0205-fb52-47a6-8a49-4e85306a521b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.401216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef0205-fb52-47a6-8a49-4e85306a521b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.401981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef0205-fb52-47a6-8a49-4e85306a521b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:24.403362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef0205-fb52-47a6-8a49-4e85306a521b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:24.420460 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fb435-78ae-40c0-ab4d-9f4f4c6834b5 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:24.429083 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 472/920] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:03:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:24.480102 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098ad8b-4270-4141-96b2-008f0b7928c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:24.480927 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ad8b-4270-4141-96b2-008f0b7928c9 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.481281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ad8b-4270-4141-96b2-008f0b7928c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.481569 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ad8b-4270-4141-96b2-008f0b7928c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.494420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098ad8b-4270-4141-96b2-008f0b7928c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e098ad8b-4270-4141-96b2-008f0b7928c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:24.503652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:24.504641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:24.505528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:24.509420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 449/921] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:03:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:24.511905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef0205-fb52-47a6-8a49-4e85306a521b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:24.621205 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef0205-fb52-47a6-8a49-4e85306a521b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:24.658642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:24.659307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.659689 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.659924 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.660680 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:24.660987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:24.662429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:24.671518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 473/922] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:03:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:24.677030 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098ad8b-4270-4141-96b2-008f0b7928c9 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:24.786271 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098ad8b-4270-4141-96b2-008f0b7928c9 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:24.819732 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ea9c4-b707-455c-8ccc-f981c0084d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:24.820523 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea9c4-b707-455c-8ccc-f981c0084d09 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.820800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea9c4-b707-455c-8ccc-f981c0084d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.821094 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ea9c4-b707-455c-8ccc-f981c0084d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.832806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ea9c4-b707-455c-8ccc-f981c0084d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:24.834295 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ea9c4-b707-455c-8ccc-f981c0084d09 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-618ea9c4-b707-455c-8ccc-f981c0084d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:24.851441 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:24.852393 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:24.853297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:24.853297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:24.853297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:24.853297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:24.853297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 450/923] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:24.859810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:24.913787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065953b1-09e4-4574-ac1e-668bf9bfddc7 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.914125 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065953b1-09e4-4574-ac1e-668bf9bfddc7 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.914344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065953b1-09e4-4574-ac1e-668bf9bfddc7 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.919478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 451/924] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:24.920916 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2d5b0-f428-4049-824c-00472e42467d None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:24.921244 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2d5b0-f428-4049-824c-00472e42467d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:24.921522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2d5b0-f428-4049-824c-00472e42467d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:24.945142 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-a7f2d5b0-f428-4049-824c-00472e42467d None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:24.946334 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-a7f2d5b0-f428-4049-824c-00472e42467d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:24.969580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45f4d-a541-499e-aaa6-9b974adcb44f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.011339 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.012355 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.012848 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.013358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.014279 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:25.014793 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.016743 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.034472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 474/925] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:25.035635 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.041353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 452/926] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:25.066613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2d5b0-f428-4049-824c-00472e42467d None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.151333 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c9dca-d023-4068-a407-9eb6c787caf2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.185446 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.186178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.186457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.186730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.195726 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.197960 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.199531 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2d5b0-f428-4049-824c-00472e42467d None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:25.215040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:25.216048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:25.216985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:25.220991 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 475/927] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:25.222209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.245063 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.246975 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.247600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.248106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.249462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.251479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:25.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:25.272485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:03:25.273383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:25.280762 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 453/928] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:25.282645 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.334405 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079897c4-b5e2-47ed-8c91-644da438478f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.379054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.380135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.380620 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.381120 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.382180 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:25.382692 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.384401 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.402082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa3082-c84f-47cb-a4ea-c3ba99a2e85b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.403941 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 476/929] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:25.413895 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.442177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.443252 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.443733 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.444248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.445191 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:25.445746 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.447339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.452782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 454/930] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:25.458740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.514267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11891a83-d9ea-462d-b859-827f80ae413b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.549252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.550064 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.550318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.550594 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.560165 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17374a7-51bf-49c3-9423-c82521f06371 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.560728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.562224 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:25.580007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:25.580990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:25.581959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:25.581959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:25.581959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:25.581959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:25.585912 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 477/931] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:25.586736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.598988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.599998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.600542 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.601013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.602025 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:25.602546 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.604768 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.614626 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 455/932] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:25.616126 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.689341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83645d00-7569-4561-a69d-bdd767e7d4d2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.719575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2577681-c96b-449d-973d-cb187ee3e1f4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.724137 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.724807 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.725117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.725398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.726129 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:25.726440 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.727877 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.746488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 478/933] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:25.747687 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.762137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1003b-8cd6-46f2-a9e0-857c802feffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.763277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1003b-8cd6-46f2-a9e0-857c802feffb None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.763784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1003b-8cd6-46f2-a9e0-857c802feffb None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.764257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1003b-8cd6-46f2-a9e0-857c802feffb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.765318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1003b-8cd6-46f2-a9e0-857c802feffb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.767003 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1003b-8cd6-46f2-a9e0-857c802feffb None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.797245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 456/934] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:03:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:03:25.859105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320261-d7bf-4db7-9d5f-83a783d317b6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:25.904944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:25.905779 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.906061 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.906330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.913948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:25.915484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:25.931298 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:25.932865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:25.934274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 479/935] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:25.937445 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf9a86-922c-4599-b151-04f7c9889d89 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.938027 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf9a86-922c-4599-b151-04f7c9889d89 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.938518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf9a86-922c-4599-b151-04f7c9889d89 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.938976 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:25.940679 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 457/936] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:25.946127 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7ec7-d29a-4b65-a690-14bf09b353b5 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:25.946409 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7ec7-d29a-4b65-a690-14bf09b353b5 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:25.947199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7ec7-d29a-4b65-a690-14bf09b353b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:25.967750 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-53fd7ec7-d29a-4b65-a690-14bf09b353b5 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:25.969676 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-53fd7ec7-d29a-4b65-a690-14bf09b353b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:26.056174 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501befd9-e730-48a4-a0a7-efff35df8fce None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:26.068001 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 458/937] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:26.090309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7ec7-d29a-4b65-a690-14bf09b353b5 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:26.098030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:26.099307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.099736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.100160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.101499 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:26.101952 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:26.104629 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:26.120567 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 480/938] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:26.127067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:26.208351 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7ec7-d29a-4b65-a690-14bf09b353b5 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:26.242365 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:26.243242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.243647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.243998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:26.246381 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:26.255938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619c803-e058-4e99-acb7-512865cadc7c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:26.259326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:26.260999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:26.261974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:26.264753 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 459/939] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:26.270335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:26.300911 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78391f3c-741d-4f2d-a97f-b8e25282f148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:26.302776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391f3c-741d-4f2d-a97f-b8e25282f148 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.303266 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391f3c-741d-4f2d-a97f-b8e25282f148 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.303766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391f3c-741d-4f2d-a97f-b8e25282f148 None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/c2567a28c4854d8191aa49947b0a82b8/roles/fa08f0adc60e48b192f5e9adecef99a5` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.307736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78391f3c-741d-4f2d-a97f-b8e25282f148 None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=c2567a28c4854d8191aa49947b0a82b8, role_id=fa08f0adc60e48b192f5e9adecef99a5)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:26.309485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78391f3c-741d-4f2d-a97f-b8e25282f148 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:26.337067 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 481/940] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:03:26 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/c2567a28c4854d8191aa49947b0a82b8/roles/fa08f0adc60e48b192f5e9adecef99a5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:03:26.386772 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630146f8-e13f-4fa9-89d4-8ff486f2de77 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:26.423115 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:26.423779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.424055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.424327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.425121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:26.425429 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:26.426795 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:26.432617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 460/941] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:26.438548 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78391f3c-741d-4f2d-a97f-b8e25282f148 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:26.543745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78391f3c-741d-4f2d-a97f-b8e25282f148 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:26.582079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:26.582745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.583021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.583292 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.584002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:26.584311 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:26.585710 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:26.591618 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 482/942] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:26.597839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:26.706820 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef04da4-6acb-43c8-8529-0f24b82539a6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:26.739867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ff494-6ba4-4ebe-8805-a620a29ced79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:26.740524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff494-6ba4-4ebe-8805-a620a29ced79 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.740806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff494-6ba4-4ebe-8805-a620a29ced79 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.741112 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ff494-6ba4-4ebe-8805-a620a29ced79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.741911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ff494-6ba4-4ebe-8805-a620a29ced79 None admin] RBAC: Adding query filter params (interface=public, service_id=f9b3a2ed4ea7477780c55b36bd801738, region_id=RegionOne) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:26.742221 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ff494-6ba4-4ebe-8805-a620a29ced79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:26.743576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ff494-6ba4-4ebe-8805-a620a29ced79 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:26.749841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 461/943] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:03:26 2022] GET /identity/v3/endpoints?service_id=f9b3a2ed4ea7477780c55b36bd801738&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:26.755549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:26.819693 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48df6-9e53-4f9a-a876-f0568bc19dc2 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.819980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48df6-9e53-4f9a-a876-f0568bc19dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.820175 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48df6-9e53-4f9a-a876-f0568bc19dc2 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.821790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 462/944] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:26.830623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce37f5f-ee75-4eac-b7e3-e14f089e1b39 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.831145 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce37f5f-ee75-4eac-b7e3-e14f089e1b39 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.831633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce37f5f-ee75-4eac-b7e3-e14f089e1b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.848498 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-3ce37f5f-ee75-4eac-b7e3-e14f089e1b39 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:26.849993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-3ce37f5f-ee75-4eac-b7e3-e14f089e1b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:26.873084 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ec771-8bdc-4c35-9334-8230555a879e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:26.912521 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0965d-a5a7-4e8d-bf0e-e377277950fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:26.913392 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0965d-a5a7-4e8d-bf0e-e377277950fe None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:26.913683 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0965d-a5a7-4e8d-bf0e-e377277950fe None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:26.913958 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0965d-a5a7-4e8d-bf0e-e377277950fe None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:26.914956 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0965d-a5a7-4e8d-bf0e-e377277950fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:26.917810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0965d-a5a7-4e8d-bf0e-e377277950fe None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:26.925917 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 483/945] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:03:26 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:03:26.926894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 463/946] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:26.950920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0965d-a5a7-4e8d-bf0e-e377277950fe None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:27.066736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0965d-a5a7-4e8d-bf0e-e377277950fe None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:27.105508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:27.106541 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:27.107054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:27.107526 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:27.108454 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:27.110178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:27.111796 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: WARNING keystone.server.flask.common [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 15:03:27.137535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.sql.core [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:03:27.137535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 15:03:27.137535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [parameters: {'id': '279796c2343b4170aa0547f97c30c3b8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 15:03:27.137535 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80062) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 279796c2343b4170aa0547f97c30c3b8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 279796c2343b4170aa0547f97c30c3b8. May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application conn.query(q) May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:03:27.145088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application result.read() May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 09 15:03:27.145929 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return project_ref.to_dict() May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application next(self.gen) May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application yield resource May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application next(self.gen) May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise self.value May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application session.commit() May 09 15:03:27.147247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.session.flush() May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application compat.raise_( May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:03:27.148690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:03:27.149728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application conn.query(q) May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application result.read() May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:03:27.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [parameters: {'id': '279796c2343b4170aa0547f97c30c3b8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:27.151731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:27.152544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.Conflict( May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 279796c2343b4170aa0547f97c30c3b8. May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:27.154691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 484/947] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 9 15:03:26 2022] POST /identity/v3/projects => generated 262 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:27.156168 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce37f5f-ee75-4eac-b7e3-e14f089e1b39 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:27.278808 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce37f5f-ee75-4eac-b7e3-e14f089e1b39 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:27.312855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb70b78-53f2-4511-a67a-b28964e6ed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:27.313669 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb70b78-53f2-4511-a67a-b28964e6ed5c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:27.313944 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb70b78-53f2-4511-a67a-b28964e6ed5c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:27.314214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb70b78-53f2-4511-a67a-b28964e6ed5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:27.320809 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb70b78-53f2-4511-a67a-b28964e6ed5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:27.322313 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb70b78-53f2-4511-a67a-b28964e6ed5c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-9cb70b78-53f2-4511-a67a-b28964e6ed5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:27.336310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:27.337413 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:27.339185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 464/948] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:27.343728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:27.446905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349d040-b59c-402b-a3e0-80c1608aff08 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:27.480398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bcfa1-14f0-49d1-9714-774ef4b262b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:27.481317 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bcfa1-14f0-49d1-9714-774ef4b262b3 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:27.481746 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bcfa1-14f0-49d1-9714-774ef4b262b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:27.482191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bcfa1-14f0-49d1-9714-774ef4b262b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:27.483096 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bcfa1-14f0-49d1-9714-774ef4b262b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:27.483544 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bcfa1-14f0-49d1-9714-774ef4b262b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:27.485436 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bcfa1-14f0-49d1-9714-774ef4b262b3 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:27.506844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 485/949] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:28.196028 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e95ea-1618-4789-bb7c-10a22d54ca63 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:28.196664 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e95ea-1618-4789-bb7c-10a22d54ca63 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:28.197207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e95ea-1618-4789-bb7c-10a22d54ca63 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:28.199036 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 465/950] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:28.204797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a279-78a1-4546-8fb3-6912eae27e8c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:28.205372 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a279-78a1-4546-8fb3-6912eae27e8c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:28.205845 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a279-78a1-4546-8fb3-6912eae27e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:28.223457 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-96e4a279-78a1-4546-8fb3-6912eae27e8c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:28.225368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-96e4a279-78a1-4546-8fb3-6912eae27e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:28.299182 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 486/951] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:28.321145 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e95ea-1618-4789-bb7c-10a22d54ca63 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:28.430410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e95ea-1618-4789-bb7c-10a22d54ca63 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:28.464133 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:28.465289 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:28.465787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:28.466273 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:28.467115 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:28.468653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:28.480714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:28.481706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:28.482549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:28.489262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 466/952] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:28.491262 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4a279-78a1-4546-8fb3-6912eae27e8c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:28.592786 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4a279-78a1-4546-8fb3-6912eae27e8c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:28.626895 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd79b3-cdca-48e8-91f5-462b2f516d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:28.627894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd79b3-cdca-48e8-91f5-462b2f516d98 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:28.628630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd79b3-cdca-48e8-91f5-462b2f516d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:28.629127 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd79b3-cdca-48e8-91f5-462b2f516d98 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:28.630048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd79b3-cdca-48e8-91f5-462b2f516d98 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:28.630547 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd79b3-cdca-48e8-91f5-462b2f516d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:28.632210 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd79b3-cdca-48e8-91f5-462b2f516d98 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:28.637540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 487/953] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:28.643215 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:28.751078 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5829f11-1b47-4c3b-aae7-bf69a28909ad None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:28.768572 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fbd61-1e2a-43b5-822b-f487d439579a None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:28.769204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fbd61-1e2a-43b5-822b-f487d439579a None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:28.769700 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fbd61-1e2a-43b5-822b-f487d439579a None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:28.771472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 488/954] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:28.777491 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971757f0-4fa6-484a-aa66-f6ef09ca3ae2 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:28.778037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971757f0-4fa6-484a-aa66-f6ef09ca3ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:28.778509 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971757f0-4fa6-484a-aa66-f6ef09ca3ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:28.788630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:28.789328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:28.789609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:28.789884 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:28.791101 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:28.791926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:28.793329 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:28.797741 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-971757f0-4fa6-484a-aa66-f6ef09ca3ae2 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:28.798634 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 467/955] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:28.805934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-971757f0-4fa6-484a-aa66-f6ef09ca3ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:28.883729 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 489/956] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:28.907132 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.014245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61ae9e-96ab-415d-99e1-c7cb29d8fa9f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.050115 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f55f3-8345-4bd1-8849-617e2f84511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.050921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f55f3-8345-4bd1-8849-617e2f84511c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.051197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f55f3-8345-4bd1-8849-617e2f84511c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.051467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f55f3-8345-4bd1-8849-617e2f84511c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.062166 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f55f3-8345-4bd1-8849-617e2f84511c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-105f55f3-8345-4bd1-8849-617e2f84511c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:29.070611 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:29.071543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:29.072436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 468/957] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:03:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:29.078884 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971757f0-4fa6-484a-aa66-f6ef09ca3ae2 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.189243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971757f0-4fa6-484a-aa66-f6ef09ca3ae2 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.225004 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06717720-6ee8-4e8c-8a48-4f15916d5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.225699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717720-6ee8-4e8c-8a48-4f15916d5abe None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.225971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717720-6ee8-4e8c-8a48-4f15916d5abe None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.227167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717720-6ee8-4e8c-8a48-4f15916d5abe None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.227167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06717720-6ee8-4e8c-8a48-4f15916d5abe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:29.227342 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06717720-6ee8-4e8c-8a48-4f15916d5abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:29.228701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06717720-6ee8-4e8c-8a48-4f15916d5abe None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:29.237273 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 490/958] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:03:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:29.242753 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f55f3-8345-4bd1-8849-617e2f84511c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.328189 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6801c-87eb-4f4e-be71-960424b1100d None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.328473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6801c-87eb-4f4e-be71-960424b1100d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.329389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6801c-87eb-4f4e-be71-960424b1100d None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.336725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 491/959] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:29.338747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4688fa-7486-47af-8cb5-3f93a76197c7 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.339264 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4688fa-7486-47af-8cb5-3f93a76197c7 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.339733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4688fa-7486-47af-8cb5-3f93a76197c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.346402 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f55f3-8345-4bd1-8849-617e2f84511c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.359021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-fe4688fa-7486-47af-8cb5-3f93a76197c7 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:29.360198 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-fe4688fa-7486-47af-8cb5-3f93a76197c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:29.382288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.383706 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.384330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.385793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.403360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:29.405832 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:29.427509 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 492/960] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:29.427841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:29.428764 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:29.429740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:29.429740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:29.429740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:29.429740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:29.429740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 469/961] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:29.435707 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4688fa-7486-47af-8cb5-3f93a76197c7 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.460309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.538483 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4688fa-7486-47af-8cb5-3f93a76197c7 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.568246 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eda49-d4bf-41fd-b1e4-667907882eac None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.573635 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.574579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.575054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.575513 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.576468 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:29.576982 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:29.578904 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:29.591166 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 493/962] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:29.597711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.609955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.610806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.611083 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.611402 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.612450 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:29.614407 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:29.630262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:29.631333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:29.632180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:29.637358 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 470/963] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:29.639642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.730669 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6bb41-6365-4165-bd1f-8ef7cc1f5315 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.755436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956e539-14ad-4f12-934b-fea746e235e4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.773978 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.776210 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.776691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.777287 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.788369 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:29.789954 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:29.793975 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.794980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.795495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.795961 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.797369 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:29.797886 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:29.799492 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:29.809087 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:29.810043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:03:29.811000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:29.811997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 494/964] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:03:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:29.812457 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 471/965] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:29.812587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.814301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.908747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbfc58-c30a-4610-8a6b-7f7f30732d80 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.924964 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b1157-9514-405c-a068-f8b2b73a4ccc None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:29.940986 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de7384-0b7c-4491-b22b-7f50d642ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.941670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de7384-0b7c-4491-b22b-7f50d642ef0e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.941947 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de7384-0b7c-4491-b22b-7f50d642ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.942219 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de7384-0b7c-4491-b22b-7f50d642ef0e None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.942915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de7384-0b7c-4491-b22b-7f50d642ef0e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:29.943217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de7384-0b7c-4491-b22b-7f50d642ef0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:29.944570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de7384-0b7c-4491-b22b-7f50d642ef0e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:29.949238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 472/966] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:29.950395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be58c-aa63-49c3-9dac-34d1a1eb6159 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.950673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be58c-aa63-49c3-9dac-34d1a1eb6159 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.950943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be58c-aa63-49c3-9dac-34d1a1eb6159 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.953069 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 473/967] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:29.954821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88be58c-aa63-49c3-9dac-34d1a1eb6159 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:29.971072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4286235-4ed6-4eb7-9ca8-f0d8a7252ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:29.972006 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286235-4ed6-4eb7-9ca8-f0d8a7252ead None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.972286 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286235-4ed6-4eb7-9ca8-f0d8a7252ead None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:29.972578 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286235-4ed6-4eb7-9ca8-f0d8a7252ead None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:29.973453 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4286235-4ed6-4eb7-9ca8-f0d8a7252ead None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:29.973761 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4286235-4ed6-4eb7-9ca8-f0d8a7252ead None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:29.975176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4286235-4ed6-4eb7-9ca8-f0d8a7252ead None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:29.997778 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 495/968] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:03:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:29.999618 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24526e5-247e-47c8-b743-86e1d07ee14e None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:29.999938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24526e5-247e-47c8-b743-86e1d07ee14e None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:30.000211 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24526e5-247e-47c8-b743-86e1d07ee14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:30.016557 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-c24526e5-247e-47c8-b743-86e1d07ee14e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:30.017742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-c24526e5-247e-47c8-b743-86e1d07ee14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:30.066687 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88be58c-aa63-49c3-9dac-34d1a1eb6159 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:30.089500 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 496/969] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:30.089500 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24526e5-247e-47c8-b743-86e1d07ee14e None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:30.102533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaf713-38e3-46b8-a9c5-04222952ffcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:30.103185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaf713-38e3-46b8-a9c5-04222952ffcc None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:30.103519 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaf713-38e3-46b8-a9c5-04222952ffcc None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:30.103793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaf713-38e3-46b8-a9c5-04222952ffcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:30.104547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaf713-38e3-46b8-a9c5-04222952ffcc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:30.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaf713-38e3-46b8-a9c5-04222952ffcc None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:30.145082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 474/970] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:03:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:03:30.145082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaf713-38e3-46b8-a9c5-04222952ffcc None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:30.243119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24526e5-247e-47c8-b743-86e1d07ee14e None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:30.254464 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaf713-38e3-46b8-a9c5-04222952ffcc None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:30.292190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:30.292863 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:30.293016 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f68cbd-75ca-47e4-8eed-26b2b55b387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:30.293187 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:30.293434 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:30.293902 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f68cbd-75ca-47e4-8eed-26b2b55b387f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:30.294139 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:30.294278 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f68cbd-75ca-47e4-8eed-26b2b55b387f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:30.294638 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f68cbd-75ca-47e4-8eed-26b2b55b387f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:30.295554 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:30.297024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: WARNING keystone.server.flask.common [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 15:03:30.312007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.sql.core [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:03:30.312007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 15:03:30.312007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [parameters: {'user_id': 'd9786cffea5c418c9262400f850c6600', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 15:03:30.312007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80060) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application conn.query(q) May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application result.read() May 09 15:03:30.335091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:03:30.336073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application next(self.gen) May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application yield resource May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application next(self.gen) May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise self.value May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application session.commit() May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:03:30.337249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self.session.flush() May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application compat.raise_( May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:03:30.339426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application result = connection._execute_20( May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:03:30.341489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application conn.query(q) May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application result.read() May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:03:30.342980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd9786cffea5c418c9262400f850c6600', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:30.347305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 09 15:03:30.349272 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 09 15:03:30.351463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 09 15:03:30.351463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 09 15:03:30.351463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:03:30.351463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:03:30.351463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 15:03:30.351463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:30.351463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 475/971] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:03:30 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:30.351463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:30.376534 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f68cbd-75ca-47e4-8eed-26b2b55b387f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:30.378223 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f68cbd-75ca-47e4-8eed-26b2b55b387f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:30.459941 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b49a-dc75-4e21-a075-35296962bb93 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:30.493708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 497/972] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:30 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:03:30.495534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:30.496435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:30.496720 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:30.496993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:30.499415 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f68cbd-75ca-47e4-8eed-26b2b55b387f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:30.505446 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:30.507093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:30.522117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:30.523119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:30.524790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 476/973] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:30.529495 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:30.606401 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f68cbd-75ca-47e4-8eed-26b2b55b387f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:30.626905 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5544d3-02d7-4df4-b270-90e6762095f9 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:30.642525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:30.643663 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:30.644150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:30.644624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:30.655907 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:30.657734 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:30.661507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:30.662172 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:30.662446 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:30.662716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:30.663515 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:30.663818 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:30.665301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:30.678705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:30.679699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:30.680659 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:30.680659 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:30.680659 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:30.680659 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:30.683566 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 498/974] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:30.690522 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:30.717151 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 477/975] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:03:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:30.801303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0377f3f-077d-4cb6-b232-ebc646bc1101 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:30.839023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:30.840040 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:30.840507 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:30.840977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:30.842011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:30.842563 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:30.844152 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:30.857554 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 499/976] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:30.863503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:30.966878 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3ecc-96a2-4cfa-ba57-656a26886d51 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:31.006310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172acf0-a191-4285-b383-e53807dca69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:31.007489 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172acf0-a191-4285-b383-e53807dca69d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:31.007957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172acf0-a191-4285-b383-e53807dca69d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:31.008430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172acf0-a191-4285-b383-e53807dca69d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:31.017488 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172acf0-a191-4285-b383-e53807dca69d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:31.019282 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172acf0-a191-4285-b383-e53807dca69d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e172acf0-a191-4285-b383-e53807dca69d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:31.034528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:31.035684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:31.036791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:31.036791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:31.036791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:31.036791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:31.036791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:31.036791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:31.036791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:31.036791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:31.038084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 478/977] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:31.043653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:31.147982 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a949-3719-42ea-8bbe-9d1627f8310c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:31.179890 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:31.180673 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:31.181074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:31.181554 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:31.182936 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:31.183347 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:31.184866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:31.191294 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9891a8-b0ce-423f-86a3-d6a7e410060d None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:31.191575 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9891a8-b0ce-423f-86a3-d6a7e410060d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:31.191841 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9891a8-b0ce-423f-86a3-d6a7e410060d None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:31.193368 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 479/978] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:31.201764 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 500/979] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:31.201908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67cfc7-dd16-44c0-b4e3-99b385d2b8fb None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:31.201908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67cfc7-dd16-44c0-b4e3-99b385d2b8fb None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:31.201908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67cfc7-dd16-44c0-b4e3-99b385d2b8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:31.203689 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:31.215476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-bd67cfc7-dd16-44c0-b4e3-99b385d2b8fb None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:31.216974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-bd67cfc7-dd16-44c0-b4e3-99b385d2b8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:31.285595 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 480/980] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:31.300637 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbed21-2531-4870-9d9d-cc16b208daae None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:31.309030 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67cfc7-dd16-44c0-b4e3-99b385d2b8fb None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:31.334015 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff278f1f-17a0-44d7-914c-681080bb6dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:31.334659 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278f1f-17a0-44d7-914c-681080bb6dba None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:31.334934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278f1f-17a0-44d7-914c-681080bb6dba None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:31.335208 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278f1f-17a0-44d7-914c-681080bb6dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:31.336023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278f1f-17a0-44d7-914c-681080bb6dba None admin] RBAC: Adding query filter params (role.id=fa08f0adc60e48b192f5e9adecef99a5, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=c2567a28c4854d8191aa49947b0a82b8) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:31.336335 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278f1f-17a0-44d7-914c-681080bb6dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:31.338361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278f1f-17a0-44d7-914c-681080bb6dba None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:31.353068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 501/981] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:31 2022] GET /identity/v3/role_assignments?user.id=c2567a28c4854d8191aa49947b0a82b8&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=fa08f0adc60e48b192f5e9adecef99a5 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:31.418695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67cfc7-dd16-44c0-b4e3-99b385d2b8fb None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:31.453956 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e794f1c-c324-4f9e-965a-6ad04a44fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:31.454633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e794f1c-c324-4f9e-965a-6ad04a44fd7a None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:31.454916 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e794f1c-c324-4f9e-965a-6ad04a44fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:31.455194 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e794f1c-c324-4f9e-965a-6ad04a44fd7a None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:31.455929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e794f1c-c324-4f9e-965a-6ad04a44fd7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:31.457354 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e794f1c-c324-4f9e-965a-6ad04a44fd7a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:31.472318 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 481/982] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:03:31 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:03:32.520526 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850b39a-edd2-46bb-b6f1-116080c22814 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:32.521100 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850b39a-edd2-46bb-b6f1-116080c22814 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:32.521100 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850b39a-edd2-46bb-b6f1-116080c22814 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:32.522501 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 502/983] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:32.528936 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9ca5-f023-4ffe-96e6-454c9298df7f None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:32.529412 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9ca5-f023-4ffe-96e6-454c9298df7f None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:32.529572 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9ca5-f023-4ffe-96e6-454c9298df7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:32.546496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-36cf9ca5-f023-4ffe-96e6-454c9298df7f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:32.547665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-36cf9ca5-f023-4ffe-96e6-454c9298df7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:32.619943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 482/984] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:32.643036 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850b39a-edd2-46bb-b6f1-116080c22814 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:32.740682 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850b39a-edd2-46bb-b6f1-116080c22814 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:32.773346 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:32.774074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:32.774355 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:32.774629 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:32.775825 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:32.777286 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:32.788984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:32.790157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:32.791054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 503/985] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:32.798211 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9ca5-f023-4ffe-96e6-454c9298df7f None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:32.890938 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9ca5-f023-4ffe-96e6-454c9298df7f None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:32.922796 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:32.923455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:32.923739 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:32.924013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:32.924721 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:32.925028 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:32.926404 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:32.936324 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 483/986] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:32.937563 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:33.034006 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a996-b890-42de-8e74-fa73a50dfa0d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:33.066380 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:33.067092 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.067379 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.067654 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.068361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:33.068666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:33.070069 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:33.079633 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 504/987] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:33.081245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:33.184692 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef0fef-5735-4739-80cb-356832206b17 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:33.218186 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f59fab-0253-46e0-a4a6-965e9b61aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:33.218851 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f59fab-0253-46e0-a4a6-965e9b61aa7c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.219132 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f59fab-0253-46e0-a4a6-965e9b61aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.219406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f59fab-0253-46e0-a4a6-965e9b61aa7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.220202 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f59fab-0253-46e0-a4a6-965e9b61aa7c None admin] RBAC: Adding query filter params (interface=admin, service_id=f9b3a2ed4ea7477780c55b36bd801738, region_id=RegionOne) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:33.220516 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f59fab-0253-46e0-a4a6-965e9b61aa7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:33.222066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f59fab-0253-46e0-a4a6-965e9b61aa7c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:33.231456 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 484/988] 10.222.0.13 () {56 vars in 1249 bytes} [Mon May 9 15:03:33 2022] GET /identity/v3/endpoints?service_id=f9b3a2ed4ea7477780c55b36bd801738&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:33.232885 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:33.332165 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182e332-d5ed-43dd-a8b7-c2ee7dbc5d37 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:33.364209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8cd2-231d-46b7-aec8-f938fc19247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:33.364872 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8cd2-231d-46b7-aec8-f938fc19247e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.365194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8cd2-231d-46b7-aec8-f938fc19247e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.365476 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8cd2-231d-46b7-aec8-f938fc19247e None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.366156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a8cd2-231d-46b7-aec8-f938fc19247e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:33.367523 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a8cd2-231d-46b7-aec8-f938fc19247e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:33.381078 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 505/989] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:03:33 2022] GET /identity/v3/services => generated 1727 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:03:33.619659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeb163-111c-4ff3-bd38-6d6aa33392f1 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.620154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeb163-111c-4ff3-bd38-6d6aa33392f1 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.620154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeb163-111c-4ff3-bd38-6d6aa33392f1 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.621774 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 485/990] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:33.627059 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1205e-d715-42e8-8378-40362aeb398a None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.627447 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1205e-d715-42e8-8378-40362aeb398a None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.627652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1205e-d715-42e8-8378-40362aeb398a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.644624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-85f1205e-d715-42e8-8378-40362aeb398a None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:33.645823 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-85f1205e-d715-42e8-8378-40362aeb398a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:33.709819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 506/991] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:33.724476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7701e-7244-4be9-b72d-90ec069feab6 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.724760 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7701e-7244-4be9-b72d-90ec069feab6 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.725058 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7701e-7244-4be9-b72d-90ec069feab6 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.726534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 486/992] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:33.732189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7701e-7244-4be9-b72d-90ec069feab6 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:33.732332 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3169-bc00-4813-9de0-5117b3856887 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.732661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3169-bc00-4813-9de0-5117b3856887 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.732850 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f3169-bc00-4813-9de0-5117b3856887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.757497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-8b5f3169-bc00-4813-9de0-5117b3856887 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:33.759199 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-8b5f3169-bc00-4813-9de0-5117b3856887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:33.830051 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 507/993] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:33.841846 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7701e-7244-4be9-b72d-90ec069feab6 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:33.852527 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f3169-bc00-4813-9de0-5117b3856887 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:33.875730 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:33.876473 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.876754 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.877323 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] PATH_INFO: `/v3/services/image` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.877702 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:33.879325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:33.890891 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:33.893228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:33.894880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 487/994] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:03:33 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:33.898673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:33.951374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f3169-bc00-4813-9de0-5117b3856887 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:33.985660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:33.986448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:33.986764 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:33.986988 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:33.987630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:33.988985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:33.995447 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba5a00-e640-4ccd-9116-75dc055e984a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:34.000508 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:34.001806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:34.002946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 508/995] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:34.007903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:34.031921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:34.032587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:34.032930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:34.033238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:34.033977 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] RBAC: Adding query filter params (name=image) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:34.034286 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:34.035639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:34.045071 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 488/996] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:34.046782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:34.104632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acae574-7631-4273-99c2-fb7dca0cdf11 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:34.148162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb02fbe-cb10-4ec9-bd9d-d55a75c9c07f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:34.148981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:34.149677 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:34.149962 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:34.150299 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:34.150984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:34.151294 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:34.152668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:34.157462 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 509/997] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:34.163009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:34.183571 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:34.184529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:34.185002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:34.185326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:34.186040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] RBAC: Adding query filter params (type=image) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:34.186397 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:34.187739 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:34.192547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 489/998] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:34.268013 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa6818-a4a6-4fa0-85e3-3ed99add13e9 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:34.303466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:34.304141 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:34.304425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:34.304701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:34.305458 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:34.305770 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:34.307752 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:34.317948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 510/999] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:34.319126 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:34.428213 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfcb15-2038-4af2-818f-b0f13953059f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:34.460313 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be472da0-4995-4098-b128-140a22270bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:34.460968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472da0-4995-4098-b128-140a22270bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:34.461268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472da0-4995-4098-b128-140a22270bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:34.461537 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472da0-4995-4098-b128-140a22270bf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:34.462326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be472da0-4995-4098-b128-140a22270bf6 None admin] RBAC: Adding query filter params (interface=public, service_id=0229539ab457490a9cda1186c050f368, region_id=RegionOne) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:34.463236 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be472da0-4995-4098-b128-140a22270bf6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:34.464614 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be472da0-4995-4098-b128-140a22270bf6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:34.474130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 490/1000] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:03:34 2022] GET /identity/v3/endpoints?service_id=0229539ab457490a9cda1186c050f368&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:34.475347 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:34.575835 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff2a15-ebc6-460a-811a-db5341d4b6ed None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:34.608187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25171b6-ad77-433b-942c-57566d30bada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:34.608856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25171b6-ad77-433b-942c-57566d30bada None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:34.609168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25171b6-ad77-433b-942c-57566d30bada None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:34.609444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25171b6-ad77-433b-942c-57566d30bada None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:34.610125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25171b6-ad77-433b-942c-57566d30bada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:34.611501 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25171b6-ad77-433b-942c-57566d30bada None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:34.621220 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 511/1001] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:03:34 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:03:35.638618 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319ba7-192e-448c-b8e2-bf26eaef9db8 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:35.639143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319ba7-192e-448c-b8e2-bf26eaef9db8 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:35.639143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b319ba7-192e-448c-b8e2-bf26eaef9db8 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:35.640653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 491/1002] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:35.645644 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e573a7-9dcb-4327-94a5-b6e7cee62a88 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:35.646033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e573a7-9dcb-4327-94a5-b6e7cee62a88 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:35.646218 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e573a7-9dcb-4327-94a5-b6e7cee62a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:35.663375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-59e573a7-9dcb-4327-94a5-b6e7cee62a88 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:35.664520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-59e573a7-9dcb-4327-94a5-b6e7cee62a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:35.726568 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 512/1003] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:35.748503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b319ba7-192e-448c-b8e2-bf26eaef9db8 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:35.847186 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b319ba7-192e-448c-b8e2-bf26eaef9db8 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:35.880197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:35.881003 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:35.881305 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:35.881581 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:35.882233 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:35.883585 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:35.895757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:35.896990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:35.898073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 492/1004] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:03:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:35.903624 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e573a7-9dcb-4327-94a5-b6e7cee62a88 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:35.997686 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e573a7-9dcb-4327-94a5-b6e7cee62a88 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:36.029870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:36.030523 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:36.030797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:36.031063 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:36.031755 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:36.032053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:36.033425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:36.043263 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 513/1005] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:36.044459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:36.140912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacecb15-6bf4-470a-8bb4-db445b874933 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:36.171635 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74071ad-eb8c-4bfd-a104-bb2c10b08d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:36.172288 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74071ad-eb8c-4bfd-a104-bb2c10b08d76 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:36.172563 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74071ad-eb8c-4bfd-a104-bb2c10b08d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:36.172831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74071ad-eb8c-4bfd-a104-bb2c10b08d76 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:36.173585 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74071ad-eb8c-4bfd-a104-bb2c10b08d76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:36.173890 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74071ad-eb8c-4bfd-a104-bb2c10b08d76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:36.175236 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74071ad-eb8c-4bfd-a104-bb2c10b08d76 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:36.184862 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 493/1006] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:03:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:36.186105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:36.281389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cacc9c-32c7-46db-af0b-ef6182c9372d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:36.316530 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4c26f-8794-4da3-887e-352aa1e15122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:36.317384 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4c26f-8794-4da3-887e-352aa1e15122 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:36.317705 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4c26f-8794-4da3-887e-352aa1e15122 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:36.317975 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4c26f-8794-4da3-887e-352aa1e15122 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:36.318918 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4c26f-8794-4da3-887e-352aa1e15122 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:36.320440 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4c26f-8794-4da3-887e-352aa1e15122 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:36.336478 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 514/1007] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:03:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:03:36.497586 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c47ff3-285b-4dce-88da-3c25db07b6ce None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:36.497930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c47ff3-285b-4dce-88da-3c25db07b6ce None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:36.498106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c47ff3-285b-4dce-88da-3c25db07b6ce None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:36.499482 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 494/1008] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:36.505561 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0dc1-942e-4fba-9983-0fa852f9c55c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:36.505863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0dc1-942e-4fba-9983-0fa852f9c55c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:36.506123 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b0dc1-942e-4fba-9983-0fa852f9c55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:36.522730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-778b0dc1-942e-4fba-9983-0fa852f9c55c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:36.523895 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-778b0dc1-942e-4fba-9983-0fa852f9c55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:36.599641 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 515/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:36.618418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c47ff3-285b-4dce-88da-3c25db07b6ce None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:36.719113 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c47ff3-285b-4dce-88da-3c25db07b6ce None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:36.751940 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9b5ee-a2df-4a43-b234-c6d8c74112d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:36.752514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9b5ee-a2df-4a43-b234-c6d8c74112d7 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:36.752789 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9b5ee-a2df-4a43-b234-c6d8c74112d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:36.753453 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9b5ee-a2df-4a43-b234-c6d8c74112d7 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:36.754177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9b5ee-a2df-4a43-b234-c6d8c74112d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:36.755523 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9b5ee-a2df-4a43-b234-c6d8c74112d7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:36.769910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 495/1010] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:03:36 2022] POST /identity/v3/services => generated 242 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:36.886844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3e10b-bbaf-4212-b39f-b5e289d34c14 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:36.887547 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3e10b-bbaf-4212-b39f-b5e289d34c14 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:36.887547 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3e10b-bbaf-4212-b39f-b5e289d34c14 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:36.888795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 516/1011] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:36.894589 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8473fb4-5b70-4ff0-8054-e5415d51023c None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:36.894872 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8473fb4-5b70-4ff0-8054-e5415d51023c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:36.895152 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8473fb4-5b70-4ff0-8054-e5415d51023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:36.913337 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-b8473fb4-5b70-4ff0-8054-e5415d51023c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:36.914541 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-b8473fb4-5b70-4ff0-8054-e5415d51023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:36.990066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 496/1012] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:36 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:37.012584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3e10b-bbaf-4212-b39f-b5e289d34c14 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:37.117844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3e10b-bbaf-4212-b39f-b5e289d34c14 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:37.151907 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:37.152687 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:37.152961 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:37.153253 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:37.153898 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:37.155248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:37.167272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:37.168329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:03:37.169679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:37.173113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 517/1013] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:37.174786 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8473fb4-5b70-4ff0-8054-e5415d51023c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:37.267956 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8473fb4-5b70-4ff0-8054-e5415d51023c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:37.300499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:37.301180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:37.301460 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:37.301730 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:37.302435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:37.302739 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:37.304081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:37.309553 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 497/1014] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:37.315371 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:37.409663 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cac8f-3d8f-4a05-b213-1854bbccd451 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:37.441618 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5f14e-9514-4ed7-9c88-250ec2ab006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:37.442339 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5f14e-9514-4ed7-9c88-250ec2ab006f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:37.442682 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5f14e-9514-4ed7-9c88-250ec2ab006f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:37.443019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5f14e-9514-4ed7-9c88-250ec2ab006f None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:37.443793 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5f14e-9514-4ed7-9c88-250ec2ab006f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:37.444159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5f14e-9514-4ed7-9c88-250ec2ab006f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:37.445618 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5f14e-9514-4ed7-9c88-250ec2ab006f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:37.455764 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 518/1015] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:03:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:37.456966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:37.554755 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d612c84-1549-4c4e-a39a-d5779452fdc5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:37.588464 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd2736-2550-4e88-94a9-0864d85cee1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:37.589153 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd2736-2550-4e88-94a9-0864d85cee1e None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:37.589438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd2736-2550-4e88-94a9-0864d85cee1e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:37.589711 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd2736-2550-4e88-94a9-0864d85cee1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:37.590453 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd2736-2550-4e88-94a9-0864d85cee1e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:37.591799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd2736-2550-4e88-94a9-0864d85cee1e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:37.617345 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 498/1016] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:03:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:03:38.581046 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a69aba-ce1a-423c-9894-dbf702f2a418 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:38.581593 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a69aba-ce1a-423c-9894-dbf702f2a418 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:38.581593 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a69aba-ce1a-423c-9894-dbf702f2a418 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:38.583095 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 519/1017] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:38.588144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a7211-f115-4f24-8a7f-4bc8d40af88f None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:38.588525 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a7211-f115-4f24-8a7f-4bc8d40af88f None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:38.588684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a7211-f115-4f24-8a7f-4bc8d40af88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:38.604752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-bb3a7211-f115-4f24-8a7f-4bc8d40af88f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:38.605929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-bb3a7211-f115-4f24-8a7f-4bc8d40af88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:38.674844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 499/1018] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:38.697483 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a69aba-ce1a-423c-9894-dbf702f2a418 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:38.803958 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a69aba-ce1a-423c-9894-dbf702f2a418 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:38.838647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f5fed-0b27-4d11-94c6-30b8d8a812aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:38.839374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f5fed-0b27-4d11-94c6-30b8d8a812aa None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:38.839649 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f5fed-0b27-4d11-94c6-30b8d8a812aa None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:38.839916 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f5fed-0b27-4d11-94c6-30b8d8a812aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:38.841234 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f5fed-0b27-4d11-94c6-30b8d8a812aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:38.842718 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f5fed-0b27-4d11-94c6-30b8d8a812aa None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:38.844612 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 520/1019] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:38.850480 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a7211-f115-4f24-8a7f-4bc8d40af88f None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:38.948957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a7211-f115-4f24-8a7f-4bc8d40af88f None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:38.982242 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5568776-0ca6-4658-88f8-7f48910b0360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:38.982899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5568776-0ca6-4658-88f8-7f48910b0360 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:38.983176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5568776-0ca6-4658-88f8-7f48910b0360 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:38.983447 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5568776-0ca6-4658-88f8-7f48910b0360 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:38.984155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5568776-0ca6-4658-88f8-7f48910b0360 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:38.985586 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5568776-0ca6-4658-88f8-7f48910b0360 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:39.017471 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 500/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:03:38 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:39.026662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23a78-d9e8-4b8f-8800-5e4cc06a6b83 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.026968 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23a78-d9e8-4b8f-8800-5e4cc06a6b83 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.027230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23a78-d9e8-4b8f-8800-5e4cc06a6b83 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.028864 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 521/1021] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:39.034593 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a841b-2cf3-4e2e-a1ae-c56614354837 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.034917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a841b-2cf3-4e2e-a1ae-c56614354837 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.035155 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a841b-2cf3-4e2e-a1ae-c56614354837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.053637 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-0c8a841b-2cf3-4e2e-a1ae-c56614354837 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:39.055013 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-0c8a841b-2cf3-4e2e-a1ae-c56614354837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:39.127362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 501/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:39.150784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23a78-d9e8-4b8f-8800-5e4cc06a6b83 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:39.254584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23a78-d9e8-4b8f-8800-5e4cc06a6b83 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:39.287716 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:39.288501 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.288784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.289077 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] PATH_INFO: `/v3/services/image` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.289724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:39.291124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:39.302434 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:39.303404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:39.304368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 522/1023] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:03:39 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:39.311008 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a841b-2cf3-4e2e-a1ae-c56614354837 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:39.405260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a841b-2cf3-4e2e-a1ae-c56614354837 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:39.438273 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:39.438933 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.439212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.439485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.440189 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] RBAC: Adding query filter params (name=image) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:39.440496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:39.441879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:39.451686 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 502/1024] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:39.453270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:39.552008 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a16d-36f6-45b0-9fe5-93c9b20716ed None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:39.585277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:39.585920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.586201 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.586473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.587177 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:39.587510 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:39.588876 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:39.593814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 523/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:39.599681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:39.694700 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c4369-d9e7-4ce4-be35-391bc015c38d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:39.726876 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecced3-393a-412b-b52a-f9c185396f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:39.727533 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecced3-393a-412b-b52a-f9c185396f30 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.727812 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecced3-393a-412b-b52a-f9c185396f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.728116 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecced3-393a-412b-b52a-f9c185396f30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.728916 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecced3-393a-412b-b52a-f9c185396f30 None admin] RBAC: Adding query filter params (interface=public, service_id=57a19c815fe448a5872c963036842ebf, region_id=RegionOne) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:39.729256 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecced3-393a-412b-b52a-f9c185396f30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:39.730652 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecced3-393a-412b-b52a-f9c185396f30 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:39.740847 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 503/1026] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:03:39 2022] GET /identity/v3/endpoints?service_id=57a19c815fe448a5872c963036842ebf&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:39.742613 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:39.849192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdea9aa-d5a3-43c7-acd0-43e9524788c6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:39.884965 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f04d8f-80f8-4a60-9550-98c5c2e0d30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:39.885966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f04d8f-80f8-4a60-9550-98c5c2e0d30c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.886370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f04d8f-80f8-4a60-9550-98c5c2e0d30c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.886772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f04d8f-80f8-4a60-9550-98c5c2e0d30c None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.887806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f04d8f-80f8-4a60-9550-98c5c2e0d30c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:39.889938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f04d8f-80f8-4a60-9550-98c5c2e0d30c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:39.909335 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 524/1027] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:03:39 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:03:39.933782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6964-e853-4ac4-abed-784453f9dc24 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.934072 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6964-e853-4ac4-abed-784453f9dc24 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.934349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6964-e853-4ac4-abed-784453f9dc24 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.936069 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 504/1028] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:39.941680 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d986-4e00-4828-a033-590b99c66f77 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:39.941983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d986-4e00-4828-a033-590b99c66f77 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:39.942233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d986-4e00-4828-a033-590b99c66f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:39.960093 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-8ef5d986-4e00-4828-a033-590b99c66f77 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:39.961261 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-8ef5d986-4e00-4828-a033-590b99c66f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:40.030761 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 525/1029] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:40.054287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b6964-e853-4ac4-abed-784453f9dc24 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:40.155653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b6964-e853-4ac4-abed-784453f9dc24 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:40.189654 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453a128-d251-4b04-879f-12e9b099e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:40.190491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453a128-d251-4b04-879f-12e9b099e1bb None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:40.190776 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453a128-d251-4b04-879f-12e9b099e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:40.191050 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453a128-d251-4b04-879f-12e9b099e1bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:40.202159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453a128-d251-4b04-879f-12e9b099e1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-a453a128-d251-4b04-879f-12e9b099e1bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:40.213206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:40.214150 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:40.215339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 505/1030] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:03:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:40.220682 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5d986-4e00-4828-a033-590b99c66f77 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:40.319023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5d986-4e00-4828-a033-590b99c66f77 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:40.351597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:40.352598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:40.353642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:40.354151 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:40.355106 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:40.355637 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:40.357247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:40.370344 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 526/1031] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:03:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:40.371847 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453a128-d251-4b04-879f-12e9b099e1bb None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:40.468197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453a128-d251-4b04-879f-12e9b099e1bb None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:40.501471 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:40.502259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:40.502536 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:40.502804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:40.515071 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:40.516513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:40.534000 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:40.534918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:40.535867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:40.535867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:40.535867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:40.535867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:40.535867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 506/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:40.541799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:40.637264 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d7b34-13ae-41f9-aec5-65c675e5286f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:40.669880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:40.670532 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:40.670810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:40.671081 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:40.671800 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:40.672103 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:40.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:40.689943 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 527/1033] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:40.691216 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:40.787509 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100ed61-42a4-4ce7-8bf6-012583506167 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:40.819109 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:40.819897 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:40.820177 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:40.820450 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:40.828576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:40.830093 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:40.844285 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:40.845287 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:40.846332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:40.847662 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 507/1034] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:03:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:40.851092 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:40.955952 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d903789-bfb9-4f23-88be-a02d818c44bb None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:40.988806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfadaa9f-b6c9-4cd7-a215-b84c58986d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:40.989484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadaa9f-b6c9-4cd7-a215-b84c58986d8b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:40.989761 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadaa9f-b6c9-4cd7-a215-b84c58986d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:40.990029 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadaa9f-b6c9-4cd7-a215-b84c58986d8b None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:40.990855 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfadaa9f-b6c9-4cd7-a215-b84c58986d8b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:40.991159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfadaa9f-b6c9-4cd7-a215-b84c58986d8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:40.992558 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfadaa9f-b6c9-4cd7-a215-b84c58986d8b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:41.014901 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 528/1035] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:03:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:41.016033 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:41.125448 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b11cc-9642-47c4-8d6c-2fe91586099a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:41.162845 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5074ef5-3c2d-44b6-8ea6-c2895719b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:41.163510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5074ef5-3c2d-44b6-8ea6-c2895719b509 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.163779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5074ef5-3c2d-44b6-8ea6-c2895719b509 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.164046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5074ef5-3c2d-44b6-8ea6-c2895719b509 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.236344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5074ef5-3c2d-44b6-8ea6-c2895719b509 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:41.237801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5074ef5-3c2d-44b6-8ea6-c2895719b509 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:41.329924 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806ac4c-9fa6-4ef8-a59c-8eac67b69072 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.330213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806ac4c-9fa6-4ef8-a59c-8eac67b69072 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.330490 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806ac4c-9fa6-4ef8-a59c-8eac67b69072 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.331987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 529/1036] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:41.337641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 508/1037] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:03:41.337788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0fd2a-b61d-4d74-92c4-ba011ea08e66 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.337788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0fd2a-b61d-4d74-92c4-ba011ea08e66 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.337788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0fd2a-b61d-4d74-92c4-ba011ea08e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.338309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5074ef5-3c2d-44b6-8ea6-c2895719b509 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:41.353085 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-36f0fd2a-b61d-4d74-92c4-ba011ea08e66 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:41.354248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-36f0fd2a-b61d-4d74-92c4-ba011ea08e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:41.421078 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 530/1038] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:41.432682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5074ef5-3c2d-44b6-8ea6-c2895719b509 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:41.443055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0fd2a-b61d-4d74-92c4-ba011ea08e66 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:41.465296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:41.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.466367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.466641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.476628 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:41.478731 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:41.495988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:41.496910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:41.497874 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:41.497874 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:41.497874 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:41.497874 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:41.497874 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 509/1039] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:41.502705 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:41.543431 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0fd2a-b61d-4d74-92c4-ba011ea08e66 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:41.577258 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201302-e9f5-48f0-b0d6-4e8d18d11049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:41.578048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201302-e9f5-48f0-b0d6-4e8d18d11049 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.578328 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201302-e9f5-48f0-b0d6-4e8d18d11049 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.578597 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d201302-e9f5-48f0-b0d6-4e8d18d11049 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.579759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d201302-e9f5-48f0-b0d6-4e8d18d11049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:41.581231 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d201302-e9f5-48f0-b0d6-4e8d18d11049 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:41.583045 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 531/1040] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:41.588375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201302-e9f5-48f0-b0d6-4e8d18d11049 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:41.599440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127334e4-e653-40c6-a549-5f1b932eb747 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:41.632353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:41.633009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.633338 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.633661 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.635214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:41.635344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:41.636594 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:41.647574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 510/1041] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:41.652996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:41.686782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d201302-e9f5-48f0-b0d6-4e8d18d11049 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:41.719366 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a5679-cb50-4c26-a99a-51835271e65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:41.720034 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5679-cb50-4c26-a99a-51835271e65b None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.720316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5679-cb50-4c26-a99a-51835271e65b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.720588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5679-cb50-4c26-a99a-51835271e65b None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.721328 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a5679-cb50-4c26-a99a-51835271e65b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:41.722827 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a5679-cb50-4c26-a99a-51835271e65b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:41.743380 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 532/1042] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:03:41 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:03:41.759172 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9ff6d-ae6e-44fa-8552-13b8dbc31817 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:41.795314 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:41.796128 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.796422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.796697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.804329 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:41.805836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:41.822557 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:41.823606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:41.824538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 511/1043] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:41.829753 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a5679-cb50-4c26-a99a-51835271e65b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:41.941863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a5679-cb50-4c26-a99a-51835271e65b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:41.979841 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61920332-dfb8-425d-89dd-00145625d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:41.980529 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61920332-dfb8-425d-89dd-00145625d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:41.980811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61920332-dfb8-425d-89dd-00145625d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:41.981113 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61920332-dfb8-425d-89dd-00145625d7b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:41.981944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61920332-dfb8-425d-89dd-00145625d7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:41.982256 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61920332-dfb8-425d-89dd-00145625d7b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:41.983739 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61920332-dfb8-425d-89dd-00145625d7b2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:41.999552 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 533/1044] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:42.006295 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:42.102865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0a185-f91a-43ce-9890-e6dfca55a355 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:42.137390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8ce46-3b1e-4625-a2fd-d2bc1c9e6857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:42.138048 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8ce46-3b1e-4625-a2fd-d2bc1c9e6857 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:42.138333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8ce46-3b1e-4625-a2fd-d2bc1c9e6857 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:42.138603 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8ce46-3b1e-4625-a2fd-d2bc1c9e6857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:42.139427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8ce46-3b1e-4625-a2fd-d2bc1c9e6857 None admin] RBAC: Adding query filter params (role.id=fa08f0adc60e48b192f5e9adecef99a5, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=512c115af57c463eb1fbe636667306dd) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:42.139737 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8ce46-3b1e-4625-a2fd-d2bc1c9e6857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:42.141162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8ce46-3b1e-4625-a2fd-d2bc1c9e6857 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:42.152886 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 512/1045] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:42 2022] GET /identity/v3/role_assignments?user.id=512c115af57c463eb1fbe636667306dd&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=fa08f0adc60e48b192f5e9adecef99a5 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:42.154930 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2811de-698f-445b-be76-ab0b960f33ce None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:42.155222 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2811de-698f-445b-be76-ab0b960f33ce None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:42.155497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2811de-698f-445b-be76-ab0b960f33ce None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:42.161148 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 534/1046] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:42.163031 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e720fc-e30f-43da-b580-765ff85d1b0d None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:42.163324 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e720fc-e30f-43da-b580-765ff85d1b0d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:42.163598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e720fc-e30f-43da-b580-765ff85d1b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:42.181809 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-58e720fc-e30f-43da-b580-765ff85d1b0d None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:42.183022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-58e720fc-e30f-43da-b580-765ff85d1b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:42.257171 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 513/1047] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:42.278975 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2811de-698f-445b-be76-ab0b960f33ce None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:42.382264 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2811de-698f-445b-be76-ab0b960f33ce None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:42.414816 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:42.416133 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:42.416411 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:42.416682 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] PATH_INFO: `/v3/services/image` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:42.417358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:42.418733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:42.429938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:42.430914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:03:42.432362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:42.434031 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 535/1048] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:03:42 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:42.437624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e720fc-e30f-43da-b580-765ff85d1b0d None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:42.529731 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e720fc-e30f-43da-b580-765ff85d1b0d None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:42.569380 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:42.570039 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:42.570311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:42.570578 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:42.571281 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] RBAC: Adding query filter params (name=image) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:42.571584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:42.573510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:42.582758 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 514/1049] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:42.583934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:42.679896 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ff7a7-c695-44dd-883c-95f108e08800 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:42.711607 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84772d19-10e7-4836-be83-52fad50c827a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:42.712977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84772d19-10e7-4836-be83-52fad50c827a None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:42.713119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84772d19-10e7-4836-be83-52fad50c827a None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:42.713361 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84772d19-10e7-4836-be83-52fad50c827a None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:42.714062 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84772d19-10e7-4836-be83-52fad50c827a None admin] RBAC: Adding query filter params (type=image) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:42.714365 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84772d19-10e7-4836-be83-52fad50c827a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:42.715725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84772d19-10e7-4836-be83-52fad50c827a None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:42.724914 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 536/1050] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:42.726409 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:42.819669 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95b18f-4177-483d-9887-884b7a204b05 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:42.852217 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89554ec7-d6fa-47a4-8fd5-9aa2c7b088f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:42.852883 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89554ec7-d6fa-47a4-8fd5-9aa2c7b088f4 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:42.853188 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89554ec7-d6fa-47a4-8fd5-9aa2c7b088f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:42.853517 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89554ec7-d6fa-47a4-8fd5-9aa2c7b088f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:42.854261 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89554ec7-d6fa-47a4-8fd5-9aa2c7b088f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:42.855592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89554ec7-d6fa-47a4-8fd5-9aa2c7b088f4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:42.874570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 515/1051] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:03:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:44.049465 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02fa04-6635-454f-bfda-76e382d25b79 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.050020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02fa04-6635-454f-bfda-76e382d25b79 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.050020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02fa04-6635-454f-bfda-76e382d25b79 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.051542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 537/1052] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:44.056661 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c7f5-f46b-4d0a-b839-08589e629fb7 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.057106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c7f5-f46b-4d0a-b839-08589e629fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.057277 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775c7f5-f46b-4d0a-b839-08589e629fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.074301 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-4775c7f5-f46b-4d0a-b839-08589e629fb7 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:44.075471 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-4775c7f5-f46b-4d0a-b839-08589e629fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:44.146908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 516/1053] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:44.170600 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02fa04-6635-454f-bfda-76e382d25b79 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:44.276125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02fa04-6635-454f-bfda-76e382d25b79 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:44.309725 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1110a-e06f-42f2-b901-6d939c12d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:44.310796 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1110a-e06f-42f2-b901-6d939c12d5bb None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.311284 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1110a-e06f-42f2-b901-6d939c12d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.311777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1110a-e06f-42f2-b901-6d939c12d5bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.322723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1110a-e06f-42f2-b901-6d939c12d5bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-7ed1110a-e06f-42f2-b901-6d939c12d5bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:44.330525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:44.331466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:44.332315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:44.334546 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 538/1054] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:44.340902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775c7f5-f46b-4d0a-b839-08589e629fb7 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:44.375095 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe6bf1-0bf4-4ed9-bd5f-1afdb7b8fc9d None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.375392 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe6bf1-0bf4-4ed9-bd5f-1afdb7b8fc9d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.375648 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe6bf1-0bf4-4ed9-bd5f-1afdb7b8fc9d None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.377159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 539/1055] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:44.382026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9f90-3b3d-456a-97ff-0a453780193c None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.382318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9f90-3b3d-456a-97ff-0a453780193c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.382594 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9f90-3b3d-456a-97ff-0a453780193c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.398549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-feda9f90-3b3d-456a-97ff-0a453780193c None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:44.400232 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-feda9f90-3b3d-456a-97ff-0a453780193c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:44.435948 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775c7f5-f46b-4d0a-b839-08589e629fb7 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:44.465145 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 540/1056] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:44.468477 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:44.469222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.469514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.469784 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.470528 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:44.470834 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:44.472178 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:44.480917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 517/1057] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:44.486502 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda9f90-3b3d-456a-97ff-0a453780193c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:44.489032 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:44.584114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda9f90-3b3d-456a-97ff-0a453780193c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:44.585380 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d6e58-04e4-413d-a737-e37db174a706 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:44.618521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd3556-d3ad-4ce6-9c0d-3f58fa661c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:44.618893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cfbaf-25b0-49fb-af8c-66f5d8189e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:44.619325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd3556-d3ad-4ce6-9c0d-3f58fa661c22 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.619603 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd3556-d3ad-4ce6-9c0d-3f58fa661c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.619764 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cfbaf-25b0-49fb-af8c-66f5d8189e3e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.619859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd3556-d3ad-4ce6-9c0d-3f58fa661c22 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.620020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cfbaf-25b0-49fb-af8c-66f5d8189e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.620336 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18cfbaf-25b0-49fb-af8c-66f5d8189e3e None admin] PATH_INFO: `/v3/users/79f52c4f0a844a52bc57440e86aebb53` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.629973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd3556-d3ad-4ce6-9c0d-3f58fa661c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:44.635161 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cfbaf-25b0-49fb-af8c-66f5d8189e3e None admin] RBAC: Authorizing `identity:get_user(user_id=79f52c4f0a844a52bc57440e86aebb53)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:44.636611 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18cfbaf-25b0-49fb-af8c-66f5d8189e3e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:44.638497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 541/1058] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/users/79f52c4f0a844a52bc57440e86aebb53 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-31cd3556-d3ad-4ce6-9c0d-3f58fa661c22 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:44.638682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:44.639601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:44.645591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 518/1059] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:44.646302 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cfbaf-25b0-49fb-af8c-66f5d8189e3e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:44.646385 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd3556-d3ad-4ce6-9c0d-3f58fa661c22 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:44.740293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18cfbaf-25b0-49fb-af8c-66f5d8189e3e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:44.741578 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd3556-d3ad-4ce6-9c0d-3f58fa661c22 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:44.773826 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95d826-9279-4b1a-b827-4843b066aa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:44.774473 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95d826-9279-4b1a-b827-4843b066aa50 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.774783 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95d826-9279-4b1a-b827-4843b066aa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.775065 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95d826-9279-4b1a-b827-4843b066aa50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.775630 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:44.776511 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.776994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.777520 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.778486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:44.778864 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:44.780334 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:44.788636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 519/1060] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:44.794795 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:44.840789 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95d826-9279-4b1a-b827-4843b066aa50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:44.842221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95d826-9279-4b1a-b827-4843b066aa50 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:44.892823 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d505b2-d2d8-4ff3-8fea-80a3192e5ce2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:44.925782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:44.926440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:44.926724 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:44.927039 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:44.937802 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:44.939421 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:44.939904 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 542/1061] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:03:44.945870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95d826-9279-4b1a-b827-4843b066aa50 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:44.959826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:44.960853 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:44.961818 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:44.961818 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:44.961818 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:44.961818 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:44.965451 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 520/1062] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:44.967466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.046455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95d826-9279-4b1a-b827-4843b066aa50 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.070229 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60776e-5329-48cf-bef1-7960a8c6b9ae None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.080308 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270c1e6-00e9-4719-a795-5834f277c3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.081198 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270c1e6-00e9-4719-a795-5834f277c3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.081479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270c1e6-00e9-4719-a795-5834f277c3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.081752 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270c1e6-00e9-4719-a795-5834f277c3d5 None admin] PATH_INFO: `/v3/projects/9a052c9192b34e6b91afb38f9272e6b1` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.082937 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270c1e6-00e9-4719-a795-5834f277c3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=9a052c9192b34e6b91afb38f9272e6b1)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.084374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270c1e6-00e9-4719-a795-5834f277c3d5 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.091549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 543/1063] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/projects/9a052c9192b34e6b91afb38f9272e6b1 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:45.092131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270c1e6-00e9-4719-a795-5834f277c3d5 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.110380 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.111046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.111325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.111598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.113120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:45.113120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.115460 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.126955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 521/1064] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:45.132681 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.189455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270c1e6-00e9-4719-a795-5834f277c3d5 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.222630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475fc9c-ef48-4ac4-9f6e-61c3f4f202c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.223270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475fc9c-ef48-4ac4-9f6e-61c3f4f202c8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.223550 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475fc9c-ef48-4ac4-9f6e-61c3f4f202c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.223824 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475fc9c-ef48-4ac4-9f6e-61c3f4f202c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.224648 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475fc9c-ef48-4ac4-9f6e-61c3f4f202c8 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=9a052c9192b34e6b91afb38f9272e6b1, user.id=79f52c4f0a844a52bc57440e86aebb53) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:45.224959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475fc9c-ef48-4ac4-9f6e-61c3f4f202c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.226472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475fc9c-ef48-4ac4-9f6e-61c3f4f202c8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.227237 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d5f0b-dc64-4aed-8e3e-9aff516b49c8 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.232276 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 544/1065] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/role_assignments?user.id=79f52c4f0a844a52bc57440e86aebb53&scope.project.id=9a052c9192b34e6b91afb38f9272e6b1&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:45.233495 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58460937-5c32-4812-bfcf-39a9642bc39f None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.233775 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58460937-5c32-4812-bfcf-39a9642bc39f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.234046 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58460937-5c32-4812-bfcf-39a9642bc39f None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.235451 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 545/1066] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:45.241086 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0439985-9624-46fd-8de9-8989dde8c0b4 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.241377 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0439985-9624-46fd-8de9-8989dde8c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.241654 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0439985-9624-46fd-8de9-8989dde8c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.260374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-a0439985-9624-46fd-8de9-8989dde8c0b4 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:45.261632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-a0439985-9624-46fd-8de9-8989dde8c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:45.264887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.265741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.266019 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.266292 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.274236 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.275734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:45.291080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:45.292333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:45.293670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 522/1067] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:45.298357 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.338204 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 546/1068] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:45.362469 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0439985-9624-46fd-8de9-8989dde8c0b4 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.403130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dbde2-d6df-43a7-886b-851b476b96f0 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.441605 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.442276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.442554 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.442826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.443663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:45.443973 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.445852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.467523 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 523/1069] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:45.469246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0439985-9624-46fd-8de9-8989dde8c0b4 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.472789 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.503944 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5388dc-63f3-459e-8cc8-2dd9dbd18dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.504589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5388dc-63f3-459e-8cc8-2dd9dbd18dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.504870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5388dc-63f3-459e-8cc8-2dd9dbd18dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.505254 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5388dc-63f3-459e-8cc8-2dd9dbd18dd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.568220 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed6fd2-2443-4365-9d0e-9a6b8da08915 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.570486 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5388dc-63f3-459e-8cc8-2dd9dbd18dd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.572221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5388dc-63f3-459e-8cc8-2dd9dbd18dd8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.601524 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.602304 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.602588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.603386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.615206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.616593 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:45.634964 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:45.635928 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:45.636855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:45.636855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:45.636855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:45.636855 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:45.640885 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 524/1070] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:45.641602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.671335 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 547/1071] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:03:45.676762 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5388dc-63f3-459e-8cc8-2dd9dbd18dd8 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.740633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f63b0-7f14-4ac2-b857-7b5291dc6273 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.771969 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5388dc-63f3-459e-8cc8-2dd9dbd18dd8 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.773867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.774530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.775367 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.775639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.776362 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:45.776665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.778057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.793726 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 525/1072] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:45.794550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.807639 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.808418 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.808702 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.809017 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.819134 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.820579 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:45.838370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:45.839575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:45.840819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:45.840819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:45.840819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:45.840819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:45.840819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 548/1073] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:45.845238 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.894212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5796f-9e7f-4ba4-97d7-c4c53a0eb001 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.930669 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.932084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.932358 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.932624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.939370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.940806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.946081 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288e5c5-3343-45fc-8d57-69ecbc2160be None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:45.955227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:45.956296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 526/1074] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:45.962388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:45.982948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:45.983612 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:45.983896 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:45.984171 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:45.984896 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:45.985235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:45.986604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:45.998493 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 549/1075] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:46.064790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25316ea4-99ec-4070-9e04-21566ec6a495 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:46.104876 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a82fe-b371-484b-a639-4779b3988f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:46.105572 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a82fe-b371-484b-a639-4779b3988f6a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:46.105831 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a82fe-b371-484b-a639-4779b3988f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:46.106193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a82fe-b371-484b-a639-4779b3988f6a None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:46.106898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a82fe-b371-484b-a639-4779b3988f6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:46.107208 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a82fe-b371-484b-a639-4779b3988f6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:46.108647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a82fe-b371-484b-a639-4779b3988f6a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:46.126349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 527/1076] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:46.127775 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:46.236021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9dbe9-72f4-4de4-b1d8-db23f5df9661 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:46.269765 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddb13b-4b84-45e7-8e89-44bb9d2a7095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:46.270998 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb13b-4b84-45e7-8e89-44bb9d2a7095 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:46.271488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb13b-4b84-45e7-8e89-44bb9d2a7095 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:46.271763 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb13b-4b84-45e7-8e89-44bb9d2a7095 None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/512c115af57c463eb1fbe636667306dd/roles/fa08f0adc60e48b192f5e9adecef99a5` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:46.273968 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddb13b-4b84-45e7-8e89-44bb9d2a7095 None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=512c115af57c463eb1fbe636667306dd, role_id=fa08f0adc60e48b192f5e9adecef99a5)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:46.276009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddb13b-4b84-45e7-8e89-44bb9d2a7095 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:46.289622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 550/1077] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:03:46 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/512c115af57c463eb1fbe636667306dd/roles/fa08f0adc60e48b192f5e9adecef99a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:03:47.473144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df62f-8143-47a4-b6fe-846083b4fe44 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:47.473663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df62f-8143-47a4-b6fe-846083b4fe44 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:47.473752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df62f-8143-47a4-b6fe-846083b4fe44 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:47.475186 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 528/1078] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:47.480721 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8680-1010-4ccc-81da-6957da1de211 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:47.481128 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8680-1010-4ccc-81da-6957da1de211 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:47.481405 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8680-1010-4ccc-81da-6957da1de211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:47.498422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-201d8680-1010-4ccc-81da-6957da1de211 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:47.499573 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-201d8680-1010-4ccc-81da-6957da1de211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:47.562079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 551/1079] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:47.583799 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12df62f-8143-47a4-b6fe-846083b4fe44 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:47.680077 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12df62f-8143-47a4-b6fe-846083b4fe44 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:47.711184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2b80-73b5-4bb5-8b41-32a42f890f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:47.711951 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2b80-73b5-4bb5-8b41-32a42f890f8c None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:47.712253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2b80-73b5-4bb5-8b41-32a42f890f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:47.712521 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2b80-73b5-4bb5-8b41-32a42f890f8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:47.722289 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac2b80-73b5-4bb5-8b41-32a42f890f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-0cac2b80-73b5-4bb5-8b41-32a42f890f8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:47.730299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:47.731395 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:47.732269 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 529/1080] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:47.737303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d8680-1010-4ccc-81da-6957da1de211 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:47.830620 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d8680-1010-4ccc-81da-6957da1de211 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:47.862711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:47.863367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:47.863645 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:47.863917 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:47.864747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:47.865068 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:47.866429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:47.874750 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 552/1081] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:47.880806 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2b80-73b5-4bb5-8b41-32a42f890f8c None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:47.974263 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2b80-73b5-4bb5-8b41-32a42f890f8c None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:48.005729 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b6be9-197e-41dd-9341-270c1beccec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:48.006518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b6be9-197e-41dd-9341-270c1beccec5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:48.006798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b6be9-197e-41dd-9341-270c1beccec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:48.007070 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b6be9-197e-41dd-9341-270c1beccec5 None admin] PATH_INFO: `/v3/users/79f52c4f0a844a52bc57440e86aebb53` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:48.008230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b6be9-197e-41dd-9341-270c1beccec5 None admin] RBAC: Authorizing `identity:get_user(user_id=79f52c4f0a844a52bc57440e86aebb53)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:48.010165 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9b6be9-197e-41dd-9341-270c1beccec5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:48.011792 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 530/1082] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:03:47 2022] GET /identity/v3/users/79f52c4f0a844a52bc57440e86aebb53 => generated 285 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:48.017341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:48.111467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b972e-732c-494f-80ba-9eb7ffd8cb10 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:48.147543 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e4748-f5c2-4217-b8a4-25983d7bb11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:48.148389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e4748-f5c2-4217-b8a4-25983d7bb11d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:48.148665 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e4748-f5c2-4217-b8a4-25983d7bb11d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:48.148935 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e4748-f5c2-4217-b8a4-25983d7bb11d None admin] PATH_INFO: `/v3/projects/9a052c9192b34e6b91afb38f9272e6b1` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:48.150163 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e4748-f5c2-4217-b8a4-25983d7bb11d None admin] RBAC: Authorizing `identity:get_project(project_id=9a052c9192b34e6b91afb38f9272e6b1)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:48.151583 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e4748-f5c2-4217-b8a4-25983d7bb11d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:48.153455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 553/1083] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:03:48 2022] GET /identity/v3/projects/9a052c9192b34e6b91afb38f9272e6b1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:48.159271 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b6be9-197e-41dd-9341-270c1beccec5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:48.255927 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b6be9-197e-41dd-9341-270c1beccec5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:48.288436 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a356b-b5e1-484a-a1c6-09f26bc11d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:48.289349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a356b-b5e1-484a-a1c6-09f26bc11d00 None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:48.289633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a356b-b5e1-484a-a1c6-09f26bc11d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:48.289909 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a356b-b5e1-484a-a1c6-09f26bc11d00 None admin] PATH_INFO: `/v3/projects/9a052c9192b34e6b91afb38f9272e6b1/users/79f52c4f0a844a52bc57440e86aebb53/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:48.292400 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a356b-b5e1-484a-a1c6-09f26bc11d00 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a052c9192b34e6b91afb38f9272e6b1, user_id=79f52c4f0a844a52bc57440e86aebb53, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:48.293859 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a356b-b5e1-484a-a1c6-09f26bc11d00 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:48.307603 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 531/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:03:48 2022] PUT /identity/v3/projects/9a052c9192b34e6b91afb38f9272e6b1/users/79f52c4f0a844a52bc57440e86aebb53/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:03:48.521779 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b0f4-86c9-4709-aa46-a9a4037c33cb None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:48.522272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b0f4-86c9-4709-aa46-a9a4037c33cb None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:48.522272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b0f4-86c9-4709-aa46-a9a4037c33cb None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:48.523599 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 554/1085] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:48.529538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23ba6-563c-4113-9e8f-578c145ed548 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:48.529925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23ba6-563c-4113-9e8f-578c145ed548 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:48.530075 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac23ba6-563c-4113-9e8f-578c145ed548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:48.546076 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-dac23ba6-563c-4113-9e8f-578c145ed548 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:48.547238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-dac23ba6-563c-4113-9e8f-578c145ed548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:48.609658 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 532/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:48.633482 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43b0f4-86c9-4709-aa46-a9a4037c33cb None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:48.730233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43b0f4-86c9-4709-aa46-a9a4037c33cb None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:48.762004 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802fe6f-c689-48a8-a1ac-278ddc9e75b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:48.762792 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802fe6f-c689-48a8-a1ac-278ddc9e75b2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:48.763072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802fe6f-c689-48a8-a1ac-278ddc9e75b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:48.763347 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802fe6f-c689-48a8-a1ac-278ddc9e75b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:48.773716 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802fe6f-c689-48a8-a1ac-278ddc9e75b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-1802fe6f-c689-48a8-a1ac-278ddc9e75b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:48.782270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:48.783196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:03:48.784057 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:48.785385 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 555/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:03:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:48.789322 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac23ba6-563c-4113-9e8f-578c145ed548 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:48.880762 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac23ba6-563c-4113-9e8f-578c145ed548 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:48.912499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:48.913185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:48.913465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:48.913736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:48.914476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:48.914782 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:48.916134 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:48.923620 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 533/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:03:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:48.929218 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802fe6f-c689-48a8-a1ac-278ddc9e75b2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:49.022281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802fe6f-c689-48a8-a1ac-278ddc9e75b2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:49.053587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:49.054283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:49.054560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:49.054828 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:49.064717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:49.066178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:49.083246 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:49.084316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:49.085404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:49.085404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:49.085404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:49.085404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:49.090111 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 556/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:49.090249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:49.194828 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d9310-d91f-4300-b51a-49b7f8c6c496 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:49.226453 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:49.227110 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:49.227387 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:49.228159 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:49.228937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:49.229267 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:49.230613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:49.241684 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 534/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:49.247809 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:49.369007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091da49-42f3-4451-81ff-40b800e459e6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:49.401837 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:49.402616 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:49.402893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:49.403160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:49.411347 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:49.412803 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:03:49.427774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:49.433795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:03:49.435105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 557/1091] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:03:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:49.435975 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:49.529391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e566b98-3de2-4438-8c1d-c23658a94c3f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:49.562156 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:49.562810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:49.563084 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:49.563355 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:49.564176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:49.564480 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:49.565924 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:49.582843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 535/1092] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:03:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:49.588977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:49.697937 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff61b6-3ba1-4139-a884-6dcce6756b84 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:49.729615 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b68cdf-d1e1-4c79-92e9-5c1712772b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:49.730243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b68cdf-d1e1-4c79-92e9-5c1712772b0e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:49.730518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b68cdf-d1e1-4c79-92e9-5c1712772b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:49.730786 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b68cdf-d1e1-4c79-92e9-5c1712772b0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:49.792850 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b68cdf-d1e1-4c79-92e9-5c1712772b0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:49.794301 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b68cdf-d1e1-4c79-92e9-5c1712772b0e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:49.885906 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 558/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:03:49.891583 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:49.986382 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c847c4-1932-4715-8f28-8b7ae7fb7766 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:50.018894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:50.019676 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.019955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.020222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.029921 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:50.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:03:50.048778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:50.049915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:50.050996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:03:50.050996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:03:50.050996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:03:50.050996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:50.050996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 536/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:50.055735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b68cdf-d1e1-4c79-92e9-5c1712772b0e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:50.150937 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b68cdf-d1e1-4c79-92e9-5c1712772b0e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:50.182896 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:50.183549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.183834 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.184112 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.184835 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:50.185244 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:50.186607 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:50.202308 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 559/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:50.203600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:50.306663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fe639-438b-461e-8186-18b68ec59ed6 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:50.353915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6eed56-fbdf-4e9b-be72-1666393e45a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:50.354717 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eed56-fbdf-4e9b-be72-1666393e45a2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.354994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eed56-fbdf-4e9b-be72-1666393e45a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.355263 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6eed56-fbdf-4e9b-be72-1666393e45a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.363076 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6eed56-fbdf-4e9b-be72-1666393e45a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:50.364530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6eed56-fbdf-4e9b-be72-1666393e45a2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-9c6eed56-fbdf-4e9b-be72-1666393e45a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:03:50.379218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:50.380241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:50.381199 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 537/1096] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:03:50.387027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:50.481967 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8d008-d91a-46d7-846b-b8726ead9c2e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:50.513607 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:50.514259 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.514540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.514815 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.515603 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:50.515916 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:50.517382 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:50.523668 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82329-35e7-496a-8feb-bd8dec13c644 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.523955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82329-35e7-496a-8feb-bd8dec13c644 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.524234 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82329-35e7-496a-8feb-bd8dec13c644 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.526145 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 538/1097] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:03:50.535174 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 560/1098] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:50.535333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202f15-d229-4e32-a08f-2fafecb2ed1e None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.535333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202f15-d229-4e32-a08f-2fafecb2ed1e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.535333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202f15-d229-4e32-a08f-2fafecb2ed1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.536008 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:50.548556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-20202f15-d229-4e32-a08f-2fafecb2ed1e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:50.549748 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-20202f15-d229-4e32-a08f-2fafecb2ed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:50.611461 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 539/1099] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:50.629495 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3da61b-ff18-4d61-a7a5-080c7fdd6689 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:50.635644 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20202f15-d229-4e32-a08f-2fafecb2ed1e None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:50.661947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:50.662658 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.662888 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.663164 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.664004 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] RBAC: Adding query filter params (role.id=fa08f0adc60e48b192f5e9adecef99a5, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=512c115af57c463eb1fbe636667306dd) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:50.664317 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:50.666324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:50.673697 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 561/1100] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:50 2022] GET /identity/v3/role_assignments?user.id=512c115af57c463eb1fbe636667306dd&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=fa08f0adc60e48b192f5e9adecef99a5 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:50.736482 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20202f15-d229-4e32-a08f-2fafecb2ed1e None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:50.771297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3c24e-5981-40a1-b632-c2ac91655457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:50.772102 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3c24e-5981-40a1-b632-c2ac91655457 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.772377 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3c24e-5981-40a1-b632-c2ac91655457 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.772647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3c24e-5981-40a1-b632-c2ac91655457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.784158 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3c24e-5981-40a1-b632-c2ac91655457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-55c3c24e-5981-40a1-b632-c2ac91655457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:03:50.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:03:50.793638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:03:50.794503 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 540/1101] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:03:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:03:50.800005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:50.901353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c209c3f-5e79-4e46-a7fd-eee975a14a44 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:50.935002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:50.935664 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:50.935943 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:50.936217 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:50.936963 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:50.937300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:50.938660 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:50.946743 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 562/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:03:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:50.952631 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3c24e-5981-40a1-b632-c2ac91655457 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:51.056474 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3c24e-5981-40a1-b632-c2ac91655457 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:51.088615 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ffce1-5268-4883-8a7d-805da9feab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:51.089506 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ffce1-5268-4883-8a7d-805da9feab2d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:51.089785 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ffce1-5268-4883-8a7d-805da9feab2d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:51.090299 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ffce1-5268-4883-8a7d-805da9feab2d None admin] PATH_INFO: `/v3/users/79f52c4f0a844a52bc57440e86aebb53` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:51.091216 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ffce1-5268-4883-8a7d-805da9feab2d None admin] RBAC: Authorizing `identity:get_user(user_id=79f52c4f0a844a52bc57440e86aebb53)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:51.092655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ffce1-5268-4883-8a7d-805da9feab2d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:51.094657 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 541/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:03:50 2022] GET /identity/v3/users/79f52c4f0a844a52bc57440e86aebb53 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:03:51.100820 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:51.203105 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07da8c5-90ca-4c49-bf05-d8e3cc6accca None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:51.234974 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d43c7-61be-4792-acc9-cd2e0cc5425c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:51.235612 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d43c7-61be-4792-acc9-cd2e0cc5425c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:51.235894 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d43c7-61be-4792-acc9-cd2e0cc5425c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:51.236177 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d43c7-61be-4792-acc9-cd2e0cc5425c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:51.303413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d43c7-61be-4792-acc9-cd2e0cc5425c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:51.304856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d43c7-61be-4792-acc9-cd2e0cc5425c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:51.405141 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 563/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:03:51.408055 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ffce1-5268-4883-8a7d-805da9feab2d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:51.505455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ffce1-5268-4883-8a7d-805da9feab2d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:51.537683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff139bd-ae8a-4c7e-a67c-d838e689905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:51.538522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff139bd-ae8a-4c7e-a67c-d838e689905b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:51.538805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff139bd-ae8a-4c7e-a67c-d838e689905b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:51.539069 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff139bd-ae8a-4c7e-a67c-d838e689905b None admin] PATH_INFO: `/v3/projects/9a052c9192b34e6b91afb38f9272e6b1` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:51.540297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff139bd-ae8a-4c7e-a67c-d838e689905b None admin] RBAC: Authorizing `identity:get_project(project_id=9a052c9192b34e6b91afb38f9272e6b1)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:51.541760 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff139bd-ae8a-4c7e-a67c-d838e689905b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:51.543449 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 542/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:03:51 2022] GET /identity/v3/projects/9a052c9192b34e6b91afb38f9272e6b1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:51.549777 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d43c7-61be-4792-acc9-cd2e0cc5425c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:51.644052 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d43c7-61be-4792-acc9-cd2e0cc5425c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:51.676291 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cca9c2-1d2c-4ec4-9915-dd6d400e1ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:51.676927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cca9c2-1d2c-4ec4-9915-dd6d400e1ede None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:51.677236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cca9c2-1d2c-4ec4-9915-dd6d400e1ede None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:51.677514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cca9c2-1d2c-4ec4-9915-dd6d400e1ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:51.678338 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cca9c2-1d2c-4ec4-9915-dd6d400e1ede None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=9a052c9192b34e6b91afb38f9272e6b1, user.id=79f52c4f0a844a52bc57440e86aebb53) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:51.678650 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cca9c2-1d2c-4ec4-9915-dd6d400e1ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:51.680071 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cca9c2-1d2c-4ec4-9915-dd6d400e1ede None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:51.686266 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 564/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:51 2022] GET /identity/v3/role_assignments?user.id=79f52c4f0a844a52bc57440e86aebb53&scope.project.id=9a052c9192b34e6b91afb38f9272e6b1&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:53.893405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b364e-5714-499d-bef6-44ce26ba9a66 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:53.893963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b364e-5714-499d-bef6-44ce26ba9a66 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:53.893963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b364e-5714-499d-bef6-44ce26ba9a66 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:53.895302 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 543/1107] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:53.900874 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e620-c4d3-4602-ba42-24fff2909ad2 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:53.901330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e620-c4d3-4602-ba42-24fff2909ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:53.901440 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e620-c4d3-4602-ba42-24fff2909ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:53.918692 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-0a00e620-c4d3-4602-ba42-24fff2909ad2 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:53.919838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-0a00e620-c4d3-4602-ba42-24fff2909ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:53.981807 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 565/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:03:54.006224 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b364e-5714-499d-bef6-44ce26ba9a66 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:54.101448 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b364e-5714-499d-bef6-44ce26ba9a66 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:54.133622 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854a901-d720-4bcb-b3ba-2a60ae5fd7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:54.134497 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854a901-d720-4bcb-b3ba-2a60ae5fd7ad None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:54.134777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854a901-d720-4bcb-b3ba-2a60ae5fd7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:54.135049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854a901-d720-4bcb-b3ba-2a60ae5fd7ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:54.136326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854a901-d720-4bcb-b3ba-2a60ae5fd7ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:54.137810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854a901-d720-4bcb-b3ba-2a60ae5fd7ad None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:54.139458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 544/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:03:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:54.145195 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00e620-c4d3-4602-ba42-24fff2909ad2 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:54.238987 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00e620-c4d3-4602-ba42-24fff2909ad2 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:54.270466 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69b871-dedd-4e1f-a4fc-dbb6bd0da5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:54.271114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69b871-dedd-4e1f-a4fc-dbb6bd0da5d6 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:54.271395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69b871-dedd-4e1f-a4fc-dbb6bd0da5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:54.271670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69b871-dedd-4e1f-a4fc-dbb6bd0da5d6 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:54.272378 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69b871-dedd-4e1f-a4fc-dbb6bd0da5d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:54.273829 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69b871-dedd-4e1f-a4fc-dbb6bd0da5d6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:54.293101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 566/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:03:54 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:03:56.560836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef49d3-939d-45b3-8748-d0aa47e66ecf None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:56.561440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef49d3-939d-45b3-8748-d0aa47e66ecf None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:56.561440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef49d3-939d-45b3-8748-d0aa47e66ecf None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:56.562753 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 545/1111] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:56.567879 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb165-f78c-46f3-875a-18f47d06c3b4 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:56.568310 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb165-f78c-46f3-875a-18f47d06c3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:56.568427 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbb165-f78c-46f3-875a-18f47d06c3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:56.585152 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-64dbb165-f78c-46f3-875a-18f47d06c3b4 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:56.586314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-64dbb165-f78c-46f3-875a-18f47d06c3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:56.648099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 567/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:56.671190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef49d3-939d-45b3-8748-d0aa47e66ecf None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:56.768641 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef49d3-939d-45b3-8748-d0aa47e66ecf None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:56.799502 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b091e-1eae-46a3-ae9b-6c20f7dd45cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:56.800345 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b091e-1eae-46a3-ae9b-6c20f7dd45cd None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:56.800624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b091e-1eae-46a3-ae9b-6c20f7dd45cd None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:56.800896 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b091e-1eae-46a3-ae9b-6c20f7dd45cd None admin] PATH_INFO: `/v3/roles/88c855fc5a0a48368fb08bf18f344893` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:56.802392 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b091e-1eae-46a3-ae9b-6c20f7dd45cd None admin] RBAC: Authorizing `identity:get_role(role_id=88c855fc5a0a48368fb08bf18f344893)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:56.803738 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b091e-1eae-46a3-ae9b-6c20f7dd45cd None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:56.805149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 546/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:03:56 2022] GET /identity/v3/roles/88c855fc5a0a48368fb08bf18f344893 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:56.811135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbb165-f78c-46f3-875a-18f47d06c3b4 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:56.904251 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbb165-f78c-46f3-875a-18f47d06c3b4 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:56.935586 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a43378-c83a-4c61-a0c8-dfc17fcf13b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:56.936432 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a43378-c83a-4c61-a0c8-dfc17fcf13b9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:56.936711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a43378-c83a-4c61-a0c8-dfc17fcf13b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:56.936989 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a43378-c83a-4c61-a0c8-dfc17fcf13b9 None admin] PATH_INFO: `/v3/users/225df8ea600b4f828e3a8df56468014d` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:56.949820 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a43378-c83a-4c61-a0c8-dfc17fcf13b9 None admin] RBAC: Authorizing `identity:get_user(user_id=225df8ea600b4f828e3a8df56468014d)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:56.951245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a43378-c83a-4c61-a0c8-dfc17fcf13b9 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:56.952838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 568/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:03:56 2022] GET /identity/v3/users/225df8ea600b4f828e3a8df56468014d => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:56.958331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b091e-1eae-46a3-ae9b-6c20f7dd45cd None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:57.051912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b091e-1eae-46a3-ae9b-6c20f7dd45cd None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:57.083588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9a3f4-b391-45b4-91af-bff89f4448c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:57.084164 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9a3f4-b391-45b4-91af-bff89f4448c9 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:57.084444 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9a3f4-b391-45b4-91af-bff89f4448c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:57.084718 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9a3f4-b391-45b4-91af-bff89f4448c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:57.148148 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9a3f4-b391-45b4-91af-bff89f4448c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:57.149656 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9a3f4-b391-45b4-91af-bff89f4448c9 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:57.245588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 547/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:03:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:03:57.251411 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a43378-c83a-4c61-a0c8-dfc17fcf13b9 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:57.343811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a43378-c83a-4c61-a0c8-dfc17fcf13b9 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:57.375351 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f672ce-2736-4906-9db9-390fb4993982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:57.376185 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f672ce-2736-4906-9db9-390fb4993982 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:57.376459 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f672ce-2736-4906-9db9-390fb4993982 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:57.376726 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f672ce-2736-4906-9db9-390fb4993982 None admin] PATH_INFO: `/v3/projects/9a052c9192b34e6b91afb38f9272e6b1` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:57.378157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f672ce-2736-4906-9db9-390fb4993982 None admin] RBAC: Authorizing `identity:get_project(project_id=9a052c9192b34e6b91afb38f9272e6b1)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:57.379582 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f672ce-2736-4906-9db9-390fb4993982 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:57.381268 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 569/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:03:57 2022] GET /identity/v3/projects/9a052c9192b34e6b91afb38f9272e6b1 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:57.387857 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9a3f4-b391-45b4-91af-bff89f4448c9 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:57.481729 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9a3f4-b391-45b4-91af-bff89f4448c9 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:57.513949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395441a1-7929-4517-befe-8b43e634ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:57.514587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395441a1-7929-4517-befe-8b43e634ddec None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:57.514866 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395441a1-7929-4517-befe-8b43e634ddec None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:57.515137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395441a1-7929-4517-befe-8b43e634ddec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:57.515957 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395441a1-7929-4517-befe-8b43e634ddec None admin] RBAC: Adding query filter params (role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=9a052c9192b34e6b91afb38f9272e6b1, user.id=225df8ea600b4f828e3a8df56468014d) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:03:57.516265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395441a1-7929-4517-befe-8b43e634ddec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:57.517720 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395441a1-7929-4517-befe-8b43e634ddec None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:57.522930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 548/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:03:57 2022] GET /identity/v3/role_assignments?user.id=225df8ea600b4f828e3a8df56468014d&scope.project.id=9a052c9192b34e6b91afb38f9272e6b1&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:59.723898 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7a859-8ca2-4285-989d-9a7fe1ad97df None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:59.724431 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7a859-8ca2-4285-989d-9a7fe1ad97df None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:59.724431 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7a859-8ca2-4285-989d-9a7fe1ad97df None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:59.725831 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 570/1118] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:03:59.731585 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ee6c-006e-4c76-9840-151eb7567995 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:59.731992 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ee6c-006e-4c76-9840-151eb7567995 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:59.732135 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6ee6c-006e-4c76-9840-151eb7567995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:59.749505 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-3aa6ee6c-006e-4c76-9840-151eb7567995 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:03:59.750673 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-3aa6ee6c-006e-4c76-9840-151eb7567995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:03:59.812416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 549/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:03:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:03:59.834540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7a859-8ca2-4285-989d-9a7fe1ad97df None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:03:59.929259 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7a859-8ca2-4285-989d-9a7fe1ad97df None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:03:59.961547 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab234eb-b41b-42f6-880f-f0db19d72cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:03:59.962421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab234eb-b41b-42f6-880f-f0db19d72cae None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:03:59.962701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab234eb-b41b-42f6-880f-f0db19d72cae None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:03:59.962971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab234eb-b41b-42f6-880f-f0db19d72cae None admin] PATH_INFO: `/v3/roles/88c855fc5a0a48368fb08bf18f344893` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:03:59.964432 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab234eb-b41b-42f6-880f-f0db19d72cae None admin] RBAC: Authorizing `identity:get_role(role_id=88c855fc5a0a48368fb08bf18f344893)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:03:59.965810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab234eb-b41b-42f6-880f-f0db19d72cae None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:03:59.967157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 571/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:03:59 2022] GET /identity/v3/roles/88c855fc5a0a48368fb08bf18f344893 => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:03:59.972649 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6ee6c-006e-4c76-9840-151eb7567995 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:00.065170 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6ee6c-006e-4c76-9840-151eb7567995 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:00.096518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181340e1-ccc7-4508-a7b4-bb6ff30ad584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:00.097397 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181340e1-ccc7-4508-a7b4-bb6ff30ad584 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:00.097680 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181340e1-ccc7-4508-a7b4-bb6ff30ad584 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:00.097955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181340e1-ccc7-4508-a7b4-bb6ff30ad584 None admin] PATH_INFO: `/v3/users/225df8ea600b4f828e3a8df56468014d` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:00.099119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181340e1-ccc7-4508-a7b4-bb6ff30ad584 None admin] RBAC: Authorizing `identity:get_user(user_id=225df8ea600b4f828e3a8df56468014d)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:00.100545 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181340e1-ccc7-4508-a7b4-bb6ff30ad584 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:00.102169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 550/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:03:59 2022] GET /identity/v3/users/225df8ea600b4f828e3a8df56468014d => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:00.111283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab234eb-b41b-42f6-880f-f0db19d72cae None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:00.210118 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab234eb-b41b-42f6-880f-f0db19d72cae None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:00.241056 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9189ee6-9c39-4f66-acfc-a4241fa6c51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:00.241908 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9189ee6-9c39-4f66-acfc-a4241fa6c51f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:00.242189 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9189ee6-9c39-4f66-acfc-a4241fa6c51f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:00.242461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9189ee6-9c39-4f66-acfc-a4241fa6c51f None admin] PATH_INFO: `/v3/projects/9a052c9192b34e6b91afb38f9272e6b1` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:00.244157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9189ee6-9c39-4f66-acfc-a4241fa6c51f None admin] RBAC: Authorizing `identity:get_project(project_id=9a052c9192b34e6b91afb38f9272e6b1)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:00.245601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9189ee6-9c39-4f66-acfc-a4241fa6c51f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:00.247276 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 572/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:04:00 2022] GET /identity/v3/projects/9a052c9192b34e6b91afb38f9272e6b1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:00.253334 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181340e1-ccc7-4508-a7b4-bb6ff30ad584 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:00.344959 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181340e1-ccc7-4508-a7b4-bb6ff30ad584 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:00.376066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c0ece-f86a-436a-9b67-f666d059693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:00.376934 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c0ece-f86a-436a-9b67-f666d059693e None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:00.377241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c0ece-f86a-436a-9b67-f666d059693e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:00.377513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c0ece-f86a-436a-9b67-f666d059693e None admin] PATH_INFO: `/v3/projects/9a052c9192b34e6b91afb38f9272e6b1/users/225df8ea600b4f828e3a8df56468014d/roles/88c855fc5a0a48368fb08bf18f344893` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:00.379923 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c0ece-f86a-436a-9b67-f666d059693e None admin] RBAC: Authorizing `identity:create_grant(project_id=9a052c9192b34e6b91afb38f9272e6b1, user_id=225df8ea600b4f828e3a8df56468014d, role_id=88c855fc5a0a48368fb08bf18f344893)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:00.381348 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c0ece-f86a-436a-9b67-f666d059693e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:00.396618 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 551/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:04:00 2022] PUT /identity/v3/projects/9a052c9192b34e6b91afb38f9272e6b1/users/225df8ea600b4f828e3a8df56468014d/roles/88c855fc5a0a48368fb08bf18f344893 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:04:02.580391 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280634ec-8baa-4a42-ba97-a4e0326b9047 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:02.580916 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280634ec-8baa-4a42-ba97-a4e0326b9047 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:02.580916 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280634ec-8baa-4a42-ba97-a4e0326b9047 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:02.582413 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 573/1124] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:02.587549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7e2f-9090-4488-9f28-0867ba668d6e None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:02.587930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7e2f-9090-4488-9f28-0867ba668d6e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:02.588094 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f7e2f-9090-4488-9f28-0867ba668d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:02.604203 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-847f7e2f-9090-4488-9f28-0867ba668d6e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:02.605391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-847f7e2f-9090-4488-9f28-0867ba668d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:02.667967 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 552/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:02.691218 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280634ec-8baa-4a42-ba97-a4e0326b9047 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:02.786128 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280634ec-8baa-4a42-ba97-a4e0326b9047 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:02.817954 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0688f-8f5d-4f1a-a61c-ef1d5476b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:02.818776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0688f-8f5d-4f1a-a61c-ef1d5476b828 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:02.819051 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0688f-8f5d-4f1a-a61c-ef1d5476b828 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:02.819320 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0688f-8f5d-4f1a-a61c-ef1d5476b828 None admin] PATH_INFO: `/v3/roles/88c855fc5a0a48368fb08bf18f344893` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:02.820760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0688f-8f5d-4f1a-a61c-ef1d5476b828 None admin] RBAC: Authorizing `identity:get_role(role_id=88c855fc5a0a48368fb08bf18f344893)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:02.822192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0688f-8f5d-4f1a-a61c-ef1d5476b828 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:02.823519 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 574/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:04:02 2022] GET /identity/v3/roles/88c855fc5a0a48368fb08bf18f344893 => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:02.829237 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f7e2f-9090-4488-9f28-0867ba668d6e None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:02.920041 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f7e2f-9090-4488-9f28-0867ba668d6e None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:02.951020 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c68a4-cfc3-4508-a84d-8aac217dab54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:02.951862 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c68a4-cfc3-4508-a84d-8aac217dab54 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:02.952136 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c68a4-cfc3-4508-a84d-8aac217dab54 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:02.952406 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c68a4-cfc3-4508-a84d-8aac217dab54 None admin] PATH_INFO: `/v3/users/225df8ea600b4f828e3a8df56468014d` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:02.953576 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c68a4-cfc3-4508-a84d-8aac217dab54 None admin] RBAC: Authorizing `identity:get_user(user_id=225df8ea600b4f828e3a8df56468014d)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:02.954970 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c68a4-cfc3-4508-a84d-8aac217dab54 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:02.956541 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 553/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:04:02 2022] GET /identity/v3/users/225df8ea600b4f828e3a8df56468014d => generated 286 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:02.962021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0688f-8f5d-4f1a-a61c-ef1d5476b828 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:03.056274 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0688f-8f5d-4f1a-a61c-ef1d5476b828 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:03.087635 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a70ea-4763-47a5-aa72-69c229f8e4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:03.088244 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a70ea-4763-47a5-aa72-69c229f8e4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:03.088520 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a70ea-4763-47a5-aa72-69c229f8e4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:03.088787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a70ea-4763-47a5-aa72-69c229f8e4e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:03.152836 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a70ea-4763-47a5-aa72-69c229f8e4e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:03.154297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a70ea-4763-47a5-aa72-69c229f8e4e0 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:03.245959 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 575/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:04:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:04:03.251768 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c68a4-cfc3-4508-a84d-8aac217dab54 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:03.343737 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c68a4-cfc3-4508-a84d-8aac217dab54 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:03.374994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a16b4-0249-47f1-ac65-ed8a74f313c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:03.375826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a16b4-0249-47f1-ac65-ed8a74f313c2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:03.376107 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a16b4-0249-47f1-ac65-ed8a74f313c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:03.376379 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a16b4-0249-47f1-ac65-ed8a74f313c2 None admin] PATH_INFO: `/v3/projects/9a052c9192b34e6b91afb38f9272e6b1` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:03.377821 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a16b4-0249-47f1-ac65-ed8a74f313c2 None admin] RBAC: Authorizing `identity:get_project(project_id=9a052c9192b34e6b91afb38f9272e6b1)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:03.379238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a16b4-0249-47f1-ac65-ed8a74f313c2 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:03.380887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 554/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:04:03 2022] GET /identity/v3/projects/9a052c9192b34e6b91afb38f9272e6b1 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:03.387592 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a70ea-4763-47a5-aa72-69c229f8e4e0 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:03.479607 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a70ea-4763-47a5-aa72-69c229f8e4e0 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:03.511931 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d199f4f-c396-42b5-ba90-fdbe157df6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:03.512507 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199f4f-c396-42b5-ba90-fdbe157df6ba None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:03.512788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199f4f-c396-42b5-ba90-fdbe157df6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:03.513080 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d199f4f-c396-42b5-ba90-fdbe157df6ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:03.513906 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199f4f-c396-42b5-ba90-fdbe157df6ba None admin] RBAC: Adding query filter params (role.id=88c855fc5a0a48368fb08bf18f344893, scope.project.id=9a052c9192b34e6b91afb38f9272e6b1, user.id=225df8ea600b4f828e3a8df56468014d) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:03.514215 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199f4f-c396-42b5-ba90-fdbe157df6ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:03.515622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d199f4f-c396-42b5-ba90-fdbe157df6ba None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:03.521225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 576/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:04:03 2022] GET /identity/v3/role_assignments?user.id=225df8ea600b4f828e3a8df56468014d&scope.project.id=9a052c9192b34e6b91afb38f9272e6b1&role.id=88c855fc5a0a48368fb08bf18f344893 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:05.737679 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8be86f-6f1f-405d-8edf-dc1043f9cc84 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:05.738249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8be86f-6f1f-405d-8edf-dc1043f9cc84 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:05.738249 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8be86f-6f1f-405d-8edf-dc1043f9cc84 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:05.739581 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 555/1131] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:05.751255 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a3fd-072e-4a0d-bf0a-63ad82c928da None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:05.751708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a3fd-072e-4a0d-bf0a-63ad82c928da None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:05.751802 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6a3fd-072e-4a0d-bf0a-63ad82c928da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:05.777601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-48a6a3fd-072e-4a0d-bf0a-63ad82c928da None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:05.779293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-48a6a3fd-072e-4a0d-bf0a-63ad82c928da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:05.850294 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 577/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:05.873942 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8be86f-6f1f-405d-8edf-dc1043f9cc84 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:05.967934 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8be86f-6f1f-405d-8edf-dc1043f9cc84 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:05.999768 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8592f4-2610-4416-a4e9-88bd33d658ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:06.000466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8592f4-2610-4416-a4e9-88bd33d658ba None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:06.000740 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8592f4-2610-4416-a4e9-88bd33d658ba None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:06.001007 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8592f4-2610-4416-a4e9-88bd33d658ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:06.002170 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8592f4-2610-4416-a4e9-88bd33d658ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:06.003829 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8592f4-2610-4416-a4e9-88bd33d658ba None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:06.005422 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 556/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:04:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:06.010990 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a3fd-072e-4a0d-bf0a-63ad82c928da None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:06.109412 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6a3fd-072e-4a0d-bf0a-63ad82c928da None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:06.142820 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eac7a2-f286-413c-8cee-149a8cae6c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:06.143474 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eac7a2-f286-413c-8cee-149a8cae6c07 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:06.143774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eac7a2-f286-413c-8cee-149a8cae6c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:06.144045 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eac7a2-f286-413c-8cee-149a8cae6c07 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:06.144736 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eac7a2-f286-413c-8cee-149a8cae6c07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:06.146325 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eac7a2-f286-413c-8cee-149a8cae6c07 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:06.171159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 578/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:04:06 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:04:08.405994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fbce0-93ba-4915-ae49-944d7403768f None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:08.406515 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fbce0-93ba-4915-ae49-944d7403768f None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:08.406515 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fbce0-93ba-4915-ae49-944d7403768f None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:08.407888 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 557/1135] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:08.413076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f74afd-5fae-4fb2-80f4-9472a8af8f66 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:08.413481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f74afd-5fae-4fb2-80f4-9472a8af8f66 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:08.413623 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f74afd-5fae-4fb2-80f4-9472a8af8f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:08.431428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-b2f74afd-5fae-4fb2-80f4-9472a8af8f66 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:08.432588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-b2f74afd-5fae-4fb2-80f4-9472a8af8f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:08.494230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 579/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:08.518197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fbce0-93ba-4915-ae49-944d7403768f None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:08.612465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fbce0-93ba-4915-ae49-944d7403768f None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:08.644125 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7522-dd55-4d99-aab4-6f75e26bd8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:08.644885 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7522-dd55-4d99-aab4-6f75e26bd8fd None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:08.645186 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7522-dd55-4d99-aab4-6f75e26bd8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:08.645454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7522-dd55-4d99-aab4-6f75e26bd8fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:08.646587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7522-dd55-4d99-aab4-6f75e26bd8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:08.647999 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7522-dd55-4d99-aab4-6f75e26bd8fd None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:08.649650 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 558/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:04:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:08.655292 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f74afd-5fae-4fb2-80f4-9472a8af8f66 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:08.746811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f74afd-5fae-4fb2-80f4-9472a8af8f66 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:08.777711 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083835ff-9445-4f23-be5c-4b774c49d5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:08.778380 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083835ff-9445-4f23-be5c-4b774c49d5a1 None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:08.778659 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083835ff-9445-4f23-be5c-4b774c49d5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:08.778930 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083835ff-9445-4f23-be5c-4b774c49d5a1 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:08.779629 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083835ff-9445-4f23-be5c-4b774c49d5a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:08.781027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083835ff-9445-4f23-be5c-4b774c49d5a1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:08.799948 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 580/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:04:08 2022] POST /identity/v3/users => generated 286 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:04:11.061015 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069d8c3-91b2-42c5-9f5f-4d3a8e514f44 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:11.061570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069d8c3-91b2-42c5-9f5f-4d3a8e514f44 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:11.061727 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069d8c3-91b2-42c5-9f5f-4d3a8e514f44 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:11.063109 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 559/1139] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:11.068247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7562ef1-f196-407b-b8e0-7aadad2fdfa5 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:11.068670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7562ef1-f196-407b-b8e0-7aadad2fdfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:11.068795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7562ef1-f196-407b-b8e0-7aadad2fdfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:11.084841 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-c7562ef1-f196-407b-b8e0-7aadad2fdfa5 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:11.086020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-c7562ef1-f196-407b-b8e0-7aadad2fdfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:11.156414 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 581/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:11.182265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069d8c3-91b2-42c5-9f5f-4d3a8e514f44 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:11.279424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069d8c3-91b2-42c5-9f5f-4d3a8e514f44 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:11.311402 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f304bf-e595-47f7-b7d2-f03052afcf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:11.312120 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f304bf-e595-47f7-b7d2-f03052afcf68 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:11.312403 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f304bf-e595-47f7-b7d2-f03052afcf68 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:11.312679 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f304bf-e595-47f7-b7d2-f03052afcf68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:11.323262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f304bf-e595-47f7-b7d2-f03052afcf68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-33f304bf-e595-47f7-b7d2-f03052afcf68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:04:11.330871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:04:11.331969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:04:11.333057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 560/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:04:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:04:11.338470 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7562ef1-f196-407b-b8e0-7aadad2fdfa5 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:11.431401 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7562ef1-f196-407b-b8e0-7aadad2fdfa5 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:11.469695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:11.470353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:11.470630 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:11.470899 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:11.471635 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:11.471938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:11.473314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:11.481620 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 582/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:04:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:11.487376 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f304bf-e595-47f7-b7d2-f03052afcf68 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:11.579205 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f304bf-e595-47f7-b7d2-f03052afcf68 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:11.610634 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b385-5736-4629-8934-be319e66cd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:11.611484 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b385-5736-4629-8934-be319e66cd49 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:11.611760 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b385-5736-4629-8934-be319e66cd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:11.612029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b385-5736-4629-8934-be319e66cd49 None admin] PATH_INFO: `/v3/users/1f2cf448c30a48f295ff07108d7a15d8` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:11.624499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45b385-5736-4629-8934-be319e66cd49 None admin] RBAC: Authorizing `identity:get_user(user_id=1f2cf448c30a48f295ff07108d7a15d8)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:11.625943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45b385-5736-4629-8934-be319e66cd49 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:11.627836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 561/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:04:11 2022] GET /identity/v3/users/1f2cf448c30a48f295ff07108d7a15d8 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:11.634114 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:11.727943 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0a529-be69-48c9-a3fd-95edfa9b0059 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:11.759370 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d28ef-4de2-47fd-96d6-6fc22f608e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:11.759986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d28ef-4de2-47fd-96d6-6fc22f608e76 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:11.760261 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d28ef-4de2-47fd-96d6-6fc22f608e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:11.760531 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d28ef-4de2-47fd-96d6-6fc22f608e76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:11.831564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d28ef-4de2-47fd-96d6-6fc22f608e76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:11.833254 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d28ef-4de2-47fd-96d6-6fc22f608e76 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:11.925706 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 583/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:04:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:04:11.931538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b385-5736-4629-8934-be319e66cd49 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:12.023579 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b385-5736-4629-8934-be319e66cd49 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:12.054567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64808e7-8e1f-4b8e-8c42-17d1e76bb834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:12.055920 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64808e7-8e1f-4b8e-8c42-17d1e76bb834 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:12.056193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64808e7-8e1f-4b8e-8c42-17d1e76bb834 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:12.056459 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64808e7-8e1f-4b8e-8c42-17d1e76bb834 None admin] PATH_INFO: `/v3/projects/56fafe3f786f48babfd128eae2ddb293` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:12.057627 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64808e7-8e1f-4b8e-8c42-17d1e76bb834 None admin] RBAC: Authorizing `identity:get_project(project_id=56fafe3f786f48babfd128eae2ddb293)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:12.059024 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64808e7-8e1f-4b8e-8c42-17d1e76bb834 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:12.060665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 562/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:04:11 2022] GET /identity/v3/projects/56fafe3f786f48babfd128eae2ddb293 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:12.066679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d28ef-4de2-47fd-96d6-6fc22f608e76 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:12.162002 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d28ef-4de2-47fd-96d6-6fc22f608e76 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:12.193816 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1261be-cb30-48d1-b9f8-e9d14cefb4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:12.194448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1261be-cb30-48d1-b9f8-e9d14cefb4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:12.194728 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1261be-cb30-48d1-b9f8-e9d14cefb4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:12.194999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1261be-cb30-48d1-b9f8-e9d14cefb4d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:12.196407 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1261be-cb30-48d1-b9f8-e9d14cefb4d1 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=56fafe3f786f48babfd128eae2ddb293, user.id=1f2cf448c30a48f295ff07108d7a15d8) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:12.196713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1261be-cb30-48d1-b9f8-e9d14cefb4d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:12.198155 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1261be-cb30-48d1-b9f8-e9d14cefb4d1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:12.203289 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 584/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:04:12 2022] GET /identity/v3/role_assignments?user.id=1f2cf448c30a48f295ff07108d7a15d8&scope.project.id=56fafe3f786f48babfd128eae2ddb293&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:14.396476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee906-f5da-4d85-b85e-ec1d715d5961 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:14.396977 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee906-f5da-4d85-b85e-ec1d715d5961 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:14.396977 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ee906-f5da-4d85-b85e-ec1d715d5961 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:14.398329 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 563/1147] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:14.403497 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940d3a4-f218-47f4-a57e-fb7dd2cd32ec None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:14.403921 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940d3a4-f218-47f4-a57e-fb7dd2cd32ec None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:14.404046 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940d3a4-f218-47f4-a57e-fb7dd2cd32ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:14.420191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-b940d3a4-f218-47f4-a57e-fb7dd2cd32ec None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:14.421393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-b940d3a4-f218-47f4-a57e-fb7dd2cd32ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:14.485200 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 585/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:14.508207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ee906-f5da-4d85-b85e-ec1d715d5961 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:14.606248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ee906-f5da-4d85-b85e-ec1d715d5961 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:14.639574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4188af1-3573-41b4-9f17-cb48cfc438c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:14.640347 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4188af1-3573-41b4-9f17-cb48cfc438c2 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:14.640626 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4188af1-3573-41b4-9f17-cb48cfc438c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:14.640898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4188af1-3573-41b4-9f17-cb48cfc438c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:14.650703 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4188af1-3573-41b4-9f17-cb48cfc438c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-c4188af1-3573-41b4-9f17-cb48cfc438c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:04:14.658227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:04:14.659309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:04:14.660330 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 564/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:04:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:04:14.666260 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940d3a4-f218-47f4-a57e-fb7dd2cd32ec None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:14.759540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940d3a4-f218-47f4-a57e-fb7dd2cd32ec None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:14.791768 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:14.792425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:14.792708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:14.792984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:14.793756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:14.794063 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:14.795433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:14.803445 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 586/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:04:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:14.810049 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4188af1-3573-41b4-9f17-cb48cfc438c2 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:14.905797 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4188af1-3573-41b4-9f17-cb48cfc438c2 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:14.937946 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0d77-3a4f-41ff-af0d-45acd87fcf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:14.938790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0d77-3a4f-41ff-af0d-45acd87fcf98 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:14.939067 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0d77-3a4f-41ff-af0d-45acd87fcf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:14.939339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0d77-3a4f-41ff-af0d-45acd87fcf98 None admin] PATH_INFO: `/v3/users/1f2cf448c30a48f295ff07108d7a15d8` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:14.940679 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0d77-3a4f-41ff-af0d-45acd87fcf98 None admin] RBAC: Authorizing `identity:get_user(user_id=1f2cf448c30a48f295ff07108d7a15d8)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:14.942140 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0d77-3a4f-41ff-af0d-45acd87fcf98 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:14.943752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 565/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:04:14 2022] GET /identity/v3/users/1f2cf448c30a48f295ff07108d7a15d8 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:14.949560 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:15.046272 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7d587-4983-46d9-b8b0-6e2cb74e127e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:15.077755 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14c850-28da-4fe1-ae45-b67325969040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:15.078593 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14c850-28da-4fe1-ae45-b67325969040 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:15.078866 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14c850-28da-4fe1-ae45-b67325969040 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:15.079131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14c850-28da-4fe1-ae45-b67325969040 None admin] PATH_INFO: `/v3/projects/56fafe3f786f48babfd128eae2ddb293` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:15.080285 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14c850-28da-4fe1-ae45-b67325969040 None admin] RBAC: Authorizing `identity:get_project(project_id=56fafe3f786f48babfd128eae2ddb293)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:15.081733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14c850-28da-4fe1-ae45-b67325969040 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:15.083392 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 587/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:04:14 2022] GET /identity/v3/projects/56fafe3f786f48babfd128eae2ddb293 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:15.089824 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0d77-3a4f-41ff-af0d-45acd87fcf98 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:15.189988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0d77-3a4f-41ff-af0d-45acd87fcf98 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:15.221314 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb075b1-9a21-44c4-b8ae-b97dd2db7bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:15.222190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb075b1-9a21-44c4-b8ae-b97dd2db7bdc None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:15.222465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb075b1-9a21-44c4-b8ae-b97dd2db7bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:15.222734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb075b1-9a21-44c4-b8ae-b97dd2db7bdc None admin] PATH_INFO: `/v3/projects/56fafe3f786f48babfd128eae2ddb293/users/1f2cf448c30a48f295ff07108d7a15d8/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:15.224643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb075b1-9a21-44c4-b8ae-b97dd2db7bdc None admin] RBAC: Authorizing `identity:create_grant(project_id=56fafe3f786f48babfd128eae2ddb293, user_id=1f2cf448c30a48f295ff07108d7a15d8, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:15.226082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb075b1-9a21-44c4-b8ae-b97dd2db7bdc None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:15.238535 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 566/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:04:15 2022] PUT /identity/v3/projects/56fafe3f786f48babfd128eae2ddb293/users/1f2cf448c30a48f295ff07108d7a15d8/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:04:17.409679 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4b80-fb5f-463d-878b-4d096e497baa None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:17.410213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4b80-fb5f-463d-878b-4d096e497baa None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:17.410213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d4b80-fb5f-463d-878b-4d096e497baa None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:17.411587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 588/1154] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:17.416704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de5c8d-bf42-4f02-b0b6-67106b266f23 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:17.417147 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de5c8d-bf42-4f02-b0b6-67106b266f23 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:17.417266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de5c8d-bf42-4f02-b0b6-67106b266f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:17.433844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-44de5c8d-bf42-4f02-b0b6-67106b266f23 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:17.434991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-44de5c8d-bf42-4f02-b0b6-67106b266f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:17.496643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 567/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:17.520702 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d4b80-fb5f-463d-878b-4d096e497baa None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:17.616801 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d4b80-fb5f-463d-878b-4d096e497baa None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:17.648838 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50a4b-9bae-4127-b94c-b19de6a0ab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:17.649632 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50a4b-9bae-4127-b94c-b19de6a0ab39 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:17.649917 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50a4b-9bae-4127-b94c-b19de6a0ab39 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:17.650192 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50a4b-9bae-4127-b94c-b19de6a0ab39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:17.660233 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d50a4b-9bae-4127-b94c-b19de6a0ab39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-06d50a4b-9bae-4127-b94c-b19de6a0ab39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:04:17.668293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:04:17.669408 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:04:17.670448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 589/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:04:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:04:17.675424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de5c8d-bf42-4f02-b0b6-67106b266f23 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:17.767494 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de5c8d-bf42-4f02-b0b6-67106b266f23 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:17.799102 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:17.799747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:17.800022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:17.800293 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:17.801027 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:17.801356 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:17.802696 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:17.810309 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 568/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:04:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:17.816023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50a4b-9bae-4127-b94c-b19de6a0ab39 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:17.910588 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50a4b-9bae-4127-b94c-b19de6a0ab39 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:17.941903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1201f9-7416-4fa6-b689-907193f13379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:17.942741 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1201f9-7416-4fa6-b689-907193f13379 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:17.943023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1201f9-7416-4fa6-b689-907193f13379 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:17.943297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1201f9-7416-4fa6-b689-907193f13379 None admin] PATH_INFO: `/v3/users/1f2cf448c30a48f295ff07108d7a15d8` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:17.944472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1201f9-7416-4fa6-b689-907193f13379 None admin] RBAC: Authorizing `identity:get_user(user_id=1f2cf448c30a48f295ff07108d7a15d8)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:17.946429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1201f9-7416-4fa6-b689-907193f13379 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:17.948013 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 590/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:04:17 2022] GET /identity/v3/users/1f2cf448c30a48f295ff07108d7a15d8 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:17.953499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:18.045206 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a7161-f0ec-4067-affe-2b96e002f5a9 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:18.076770 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6df5df-06ee-453f-a547-ba2c526e2381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:18.077428 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6df5df-06ee-453f-a547-ba2c526e2381 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:18.077709 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6df5df-06ee-453f-a547-ba2c526e2381 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:18.077990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6df5df-06ee-453f-a547-ba2c526e2381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:18.141149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6df5df-06ee-453f-a547-ba2c526e2381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:18.142586 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6df5df-06ee-453f-a547-ba2c526e2381 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:18.232998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 569/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:04:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:04:18.238782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1201f9-7416-4fa6-b689-907193f13379 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:18.332603 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1201f9-7416-4fa6-b689-907193f13379 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:18.363837 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5b9e6-7607-4ddf-8bcb-c2e378fc2fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:18.364662 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b9e6-7607-4ddf-8bcb-c2e378fc2fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:18.364943 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b9e6-7607-4ddf-8bcb-c2e378fc2fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:18.365247 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5b9e6-7607-4ddf-8bcb-c2e378fc2fc8 None admin] PATH_INFO: `/v3/projects/56fafe3f786f48babfd128eae2ddb293` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:18.366411 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5b9e6-7607-4ddf-8bcb-c2e378fc2fc8 None admin] RBAC: Authorizing `identity:get_project(project_id=56fafe3f786f48babfd128eae2ddb293)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:18.368348 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5b9e6-7607-4ddf-8bcb-c2e378fc2fc8 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:18.370033 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 591/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:04:18 2022] GET /identity/v3/projects/56fafe3f786f48babfd128eae2ddb293 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:18.375800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6df5df-06ee-453f-a547-ba2c526e2381 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:18.467241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6df5df-06ee-453f-a547-ba2c526e2381 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:18.499360 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d41dff-e863-4af3-b11d-a030070f698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:18.499934 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d41dff-e863-4af3-b11d-a030070f698f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:18.500211 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d41dff-e863-4af3-b11d-a030070f698f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:18.500482 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d41dff-e863-4af3-b11d-a030070f698f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:18.501327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d41dff-e863-4af3-b11d-a030070f698f None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=56fafe3f786f48babfd128eae2ddb293, user.id=1f2cf448c30a48f295ff07108d7a15d8) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:18.501636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d41dff-e863-4af3-b11d-a030070f698f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:18.503291 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d41dff-e863-4af3-b11d-a030070f698f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:18.508809 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 570/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:04:18 2022] GET /identity/v3/role_assignments?user.id=1f2cf448c30a48f295ff07108d7a15d8&scope.project.id=56fafe3f786f48babfd128eae2ddb293&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 608 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:20.737302 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77d673-d162-4694-9351-bebafd266034 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:20.737893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77d673-d162-4694-9351-bebafd266034 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:20.737893 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77d673-d162-4694-9351-bebafd266034 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:20.739242 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 592/1162] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:20.744988 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea1364-a953-4204-a2a0-7dc41d367438 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:20.745550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea1364-a953-4204-a2a0-7dc41d367438 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:20.745550 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea1364-a953-4204-a2a0-7dc41d367438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:20.762530 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-00ea1364-a953-4204-a2a0-7dc41d367438 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:20.763720 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-00ea1364-a953-4204-a2a0-7dc41d367438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:20.833195 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 571/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:20.855664 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77d673-d162-4694-9351-bebafd266034 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:20.951559 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77d673-d162-4694-9351-bebafd266034 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:20.983922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60540394-b739-4e74-82e3-d2f4286cff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:20.984545 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60540394-b739-4e74-82e3-d2f4286cff75 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:20.984826 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60540394-b739-4e74-82e3-d2f4286cff75 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:20.985127 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60540394-b739-4e74-82e3-d2f4286cff75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:21.048125 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60540394-b739-4e74-82e3-d2f4286cff75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:21.049529 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60540394-b739-4e74-82e3-d2f4286cff75 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:21.156110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 593/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:04:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:04:21.162280 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea1364-a953-4204-a2a0-7dc41d367438 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:21.255526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea1364-a953-4204-a2a0-7dc41d367438 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:21.286968 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef80e5a-2346-4588-a212-a5b5dfcab504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:21.287752 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80e5a-2346-4588-a212-a5b5dfcab504 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:21.288029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80e5a-2346-4588-a212-a5b5dfcab504 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:21.288300 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef80e5a-2346-4588-a212-a5b5dfcab504 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:21.295278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef80e5a-2346-4588-a212-a5b5dfcab504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:21.296731 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef80e5a-2346-4588-a212-a5b5dfcab504 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-2ef80e5a-2346-4588-a212-a5b5dfcab504 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:04:21.310879 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:04:21.312091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:04:21.313389 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 572/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:04:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:04:21.317930 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60540394-b739-4e74-82e3-d2f4286cff75 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:21.410604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60540394-b739-4e74-82e3-d2f4286cff75 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:21.442797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92061622-14ed-4084-bb49-a379e407c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:21.443433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92061622-14ed-4084-bb49-a379e407c008 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:21.443713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92061622-14ed-4084-bb49-a379e407c008 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:21.443986 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92061622-14ed-4084-bb49-a379e407c008 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:21.444696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92061622-14ed-4084-bb49-a379e407c008 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:21.445000 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92061622-14ed-4084-bb49-a379e407c008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:21.446401 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92061622-14ed-4084-bb49-a379e407c008 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:21.453196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 594/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:04:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:23.626798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84e0f7-a8d3-4a04-bc84-3c54d6843be3 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:23.627607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84e0f7-a8d3-4a04-bc84-3c54d6843be3 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:23.627607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84e0f7-a8d3-4a04-bc84-3c54d6843be3 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:23.628967 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 573/1167] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:23.634622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac24b6-df82-4157-9d92-941469695902 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:23.635051 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac24b6-df82-4157-9d92-941469695902 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:23.635178 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac24b6-df82-4157-9d92-941469695902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:23.651217 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-00ac24b6-df82-4157-9d92-941469695902 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:23.652369 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-00ac24b6-df82-4157-9d92-941469695902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:23.713760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 595/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:23.736014 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84e0f7-a8d3-4a04-bc84-3c54d6843be3 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:23.831009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84e0f7-a8d3-4a04-bc84-3c54d6843be3 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:23.862056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d71e79-e308-4852-8f2c-5cdb8c3e0797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:23.862690 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71e79-e308-4852-8f2c-5cdb8c3e0797 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:23.862967 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71e79-e308-4852-8f2c-5cdb8c3e0797 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:23.863239 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71e79-e308-4852-8f2c-5cdb8c3e0797 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:23.863963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d71e79-e308-4852-8f2c-5cdb8c3e0797 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:23.865327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d71e79-e308-4852-8f2c-5cdb8c3e0797 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:23.879523 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 574/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Mon May 9 15:04:23 2022] POST /identity/v3/domains => generated 267 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:04:26.108318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94038476-58ad-4240-8557-7701825def69 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:26.108865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94038476-58ad-4240-8557-7701825def69 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:26.108865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94038476-58ad-4240-8557-7701825def69 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:26.110254 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 596/1170] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:26.115475 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0de0a-759d-46c2-b791-3f875d9fcd86 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:26.115906 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0de0a-759d-46c2-b791-3f875d9fcd86 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:26.116029 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0de0a-759d-46c2-b791-3f875d9fcd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:26.132927 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-f8a0de0a-759d-46c2-b791-3f875d9fcd86 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:26.134100 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-f8a0de0a-759d-46c2-b791-3f875d9fcd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:26.195183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 575/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:26.217935 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94038476-58ad-4240-8557-7701825def69 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:26.315095 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94038476-58ad-4240-8557-7701825def69 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:26.351014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67cc8f-f7ac-4a1b-ad27-ee4d6952c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:26.351880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67cc8f-f7ac-4a1b-ad27-ee4d6952c962 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:26.352157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67cc8f-f7ac-4a1b-ad27-ee4d6952c962 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:26.352427 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67cc8f-f7ac-4a1b-ad27-ee4d6952c962 None admin] PATH_INFO: `/v3/domains/ef4ce5fdceea4c569cec4725502be5e3` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:26.362315 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67cc8f-f7ac-4a1b-ad27-ee4d6952c962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ef4ce5fdceea4c569cec4725502be5e3)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:26.363698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67cc8f-f7ac-4a1b-ad27-ee4d6952c962 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:26.365358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 597/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 9 15:04:26 2022] GET /identity/v3/domains/ef4ce5fdceea4c569cec4725502be5e3 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:26.371613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0de0a-759d-46c2-b791-3f875d9fcd86 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:26.462804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0de0a-759d-46c2-b791-3f875d9fcd86 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:26.493789 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdeb75-3ec4-47b0-8a4c-8594b9c29c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:26.494418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdeb75-3ec4-47b0-8a4c-8594b9c29c76 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:26.494695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdeb75-3ec4-47b0-8a4c-8594b9c29c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:26.494966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdeb75-3ec4-47b0-8a4c-8594b9c29c76 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:26.495666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdeb75-3ec4-47b0-8a4c-8594b9c29c76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:26.497099 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdeb75-3ec4-47b0-8a4c-8594b9c29c76 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:26.520724 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 576/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:04:26 2022] POST /identity/v3/projects => generated 361 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:04:28.744696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3000ad-1f20-48de-a8d7-1a7d9991e2d5 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:28.745319 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3000ad-1f20-48de-a8d7-1a7d9991e2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:28.745319 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3000ad-1f20-48de-a8d7-1a7d9991e2d5 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:28.746647 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 598/1174] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:28.751876 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c44ce-7db8-4692-9841-b17fae569812 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:28.752527 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c44ce-7db8-4692-9841-b17fae569812 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:28.752670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c44ce-7db8-4692-9841-b17fae569812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:28.769264 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-c85c44ce-7db8-4692-9841-b17fae569812 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:28.770404 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-c85c44ce-7db8-4692-9841-b17fae569812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:28.836620 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 577/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:28.861330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3000ad-1f20-48de-a8d7-1a7d9991e2d5 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:28.956922 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3000ad-1f20-48de-a8d7-1a7d9991e2d5 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:28.987760 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd59ba-7895-443a-b829-ac146b995891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:28.988591 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd59ba-7895-443a-b829-ac146b995891 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:28.988865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd59ba-7895-443a-b829-ac146b995891 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:28.989158 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd59ba-7895-443a-b829-ac146b995891 None admin] PATH_INFO: `/v3/domains/ef4ce5fdceea4c569cec4725502be5e3` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:28.990310 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd59ba-7895-443a-b829-ac146b995891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ef4ce5fdceea4c569cec4725502be5e3)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:28.991714 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd59ba-7895-443a-b829-ac146b995891 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:28.993347 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 599/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 9 15:04:28 2022] GET /identity/v3/domains/ef4ce5fdceea4c569cec4725502be5e3 => generated 267 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:28.999012 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c44ce-7db8-4692-9841-b17fae569812 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:29.090514 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c44ce-7db8-4692-9841-b17fae569812 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:29.121193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de783327-dc2b-455e-b272-12a48aa21d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:29.121843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de783327-dc2b-455e-b272-12a48aa21d5e None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:29.122119 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de783327-dc2b-455e-b272-12a48aa21d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:29.122391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de783327-dc2b-455e-b272-12a48aa21d5e None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:29.123094 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de783327-dc2b-455e-b272-12a48aa21d5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:29.124494 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de783327-dc2b-455e-b272-12a48aa21d5e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:29.144656 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 578/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:04:28 2022] POST /identity/v3/users => generated 311 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:04:31.383696 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b100f-0065-4bd3-ad44-baa485956f14 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:31.384194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b100f-0065-4bd3-ad44-baa485956f14 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:31.384194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b100f-0065-4bd3-ad44-baa485956f14 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:31.385542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 600/1178] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:31.390795 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94911ae4-e0d3-4a7d-b771-dbd6d64ebe3e None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:31.391192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94911ae4-e0d3-4a7d-b771-dbd6d64ebe3e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:31.391334 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94911ae4-e0d3-4a7d-b771-dbd6d64ebe3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:31.407851 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-94911ae4-e0d3-4a7d-b771-dbd6d64ebe3e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:31.408994 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-94911ae4-e0d3-4a7d-b771-dbd6d64ebe3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:31.469666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 579/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:31.492154 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b100f-0065-4bd3-ad44-baa485956f14 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:31.587475 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b100f-0065-4bd3-ad44-baa485956f14 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:31.618257 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a552660-81de-4497-a69b-91098911d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:31.619018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a552660-81de-4497-a69b-91098911d417 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:31.619293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a552660-81de-4497-a69b-91098911d417 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:31.619562 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a552660-81de-4497-a69b-91098911d417 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:31.629666 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a552660-81de-4497-a69b-91098911d417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-0a552660-81de-4497-a69b-91098911d417 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:04:31.637766 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:04:31.638851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:04:31.639865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 601/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:04:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:04:31.644529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94911ae4-e0d3-4a7d-b771-dbd6d64ebe3e None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:31.736664 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94911ae4-e0d3-4a7d-b771-dbd6d64ebe3e None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:31.768405 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:31.769069 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:31.769349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:31.769618 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:31.770347 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:31.770648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:31.771990 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:31.779476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 580/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:31.785026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a552660-81de-4497-a69b-91098911d417 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:31.879465 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a552660-81de-4497-a69b-91098911d417 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:31.910715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eb5fc-c4d4-4cab-af55-555824531a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:31.911558 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eb5fc-c4d4-4cab-af55-555824531a8c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:31.911832 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eb5fc-c4d4-4cab-af55-555824531a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:31.912097 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eb5fc-c4d4-4cab-af55-555824531a8c None admin] PATH_INFO: `/v3/users/2fcfe533d3534423905f9121aab35d3d` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:31.925027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960eb5fc-c4d4-4cab-af55-555824531a8c None admin] RBAC: Authorizing `identity:get_user(user_id=2fcfe533d3534423905f9121aab35d3d)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:31.926482 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960eb5fc-c4d4-4cab-af55-555824531a8c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:31.928115 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 602/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:04:31 2022] GET /identity/v3/users/2fcfe533d3534423905f9121aab35d3d => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:31.934419 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:32.026045 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80138345-ab5d-496e-a196-ec255540c5cf None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:32.057308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f371b4e-e2fa-4194-b0e6-51e0a8091a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:32.057929 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f371b4e-e2fa-4194-b0e6-51e0a8091a3b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:32.058205 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f371b4e-e2fa-4194-b0e6-51e0a8091a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:32.058474 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f371b4e-e2fa-4194-b0e6-51e0a8091a3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:32.120196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f371b4e-e2fa-4194-b0e6-51e0a8091a3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:32.121688 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f371b4e-e2fa-4194-b0e6-51e0a8091a3b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:32.211322 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 581/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:04:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:04:32.217221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eb5fc-c4d4-4cab-af55-555824531a8c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:32.309773 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eb5fc-c4d4-4cab-af55-555824531a8c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:32.341028 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749eff5-71d0-4bdf-84f5-7f1e20be3a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:32.341878 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749eff5-71d0-4bdf-84f5-7f1e20be3a59 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:32.342156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749eff5-71d0-4bdf-84f5-7f1e20be3a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:32.342425 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749eff5-71d0-4bdf-84f5-7f1e20be3a59 None admin] PATH_INFO: `/v3/projects/2b40a75d89564271826427e6e9d932a5` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:32.343563 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749eff5-71d0-4bdf-84f5-7f1e20be3a59 None admin] RBAC: Authorizing `identity:get_project(project_id=2b40a75d89564271826427e6e9d932a5)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:32.344971 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749eff5-71d0-4bdf-84f5-7f1e20be3a59 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:32.346657 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 603/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:04:32 2022] GET /identity/v3/projects/2b40a75d89564271826427e6e9d932a5 => generated 361 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:32.353311 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f371b4e-e2fa-4194-b0e6-51e0a8091a3b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:32.444370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f371b4e-e2fa-4194-b0e6-51e0a8091a3b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:32.476613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1327b0-5306-4818-a9f4-5980d0c764aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:32.477240 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1327b0-5306-4818-a9f4-5980d0c764aa None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:32.477522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1327b0-5306-4818-a9f4-5980d0c764aa None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:32.477795 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1327b0-5306-4818-a9f4-5980d0c764aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:32.478619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1327b0-5306-4818-a9f4-5980d0c764aa None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=2b40a75d89564271826427e6e9d932a5, user.id=2fcfe533d3534423905f9121aab35d3d) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:32.478944 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1327b0-5306-4818-a9f4-5980d0c764aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:32.480339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1327b0-5306-4818-a9f4-5980d0c764aa None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:32.485642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 582/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:04:32 2022] GET /identity/v3/role_assignments?user.id=2fcfe533d3534423905f9121aab35d3d&scope.project.id=2b40a75d89564271826427e6e9d932a5&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 259 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:34.652498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8f4e2-4bcc-4351-91d1-a3c72923c1f3 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:34.653020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8f4e2-4bcc-4351-91d1-a3c72923c1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:34.653020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8f4e2-4bcc-4351-91d1-a3c72923c1f3 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:34.654422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 604/1186] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:34.659520 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985db358-4e47-4b4c-93e1-0b87ec2c3ffa None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:34.659904 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985db358-4e47-4b4c-93e1-0b87ec2c3ffa None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:34.660062 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985db358-4e47-4b4c-93e1-0b87ec2c3ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:34.676114 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-985db358-4e47-4b4c-93e1-0b87ec2c3ffa None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:34.677361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-985db358-4e47-4b4c-93e1-0b87ec2c3ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:34.738635 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 583/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:34.761626 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8f4e2-4bcc-4351-91d1-a3c72923c1f3 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:34.855798 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8f4e2-4bcc-4351-91d1-a3c72923c1f3 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:34.887984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b77a00-733a-4935-afbf-8d673066b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:34.888740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77a00-733a-4935-afbf-8d673066b72c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:34.889021 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77a00-733a-4935-afbf-8d673066b72c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:34.889321 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b77a00-733a-4935-afbf-8d673066b72c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:34.898945 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b77a00-733a-4935-afbf-8d673066b72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-e8b77a00-733a-4935-afbf-8d673066b72c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:04:34.906514 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:04:34.907604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:04:34.908528 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 605/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:04:34.913911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985db358-4e47-4b4c-93e1-0b87ec2c3ffa None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:35.005871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985db358-4e47-4b4c-93e1-0b87ec2c3ffa None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:35.039187 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:35.040250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:35.040610 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:35.041245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:35.042353 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:35.042750 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:35.045089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:35.053388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 584/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:35.059842 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b77a00-733a-4935-afbf-8d673066b72c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:35.152983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b77a00-733a-4935-afbf-8d673066b72c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:35.184772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad8a5-73e6-4d0e-b4c6-582a63fe617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:35.185594 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad8a5-73e6-4d0e-b4c6-582a63fe617d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:35.185869 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad8a5-73e6-4d0e-b4c6-582a63fe617d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:35.186160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad8a5-73e6-4d0e-b4c6-582a63fe617d None admin] PATH_INFO: `/v3/users/2fcfe533d3534423905f9121aab35d3d` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:35.187322 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ad8a5-73e6-4d0e-b4c6-582a63fe617d None admin] RBAC: Authorizing `identity:get_user(user_id=2fcfe533d3534423905f9121aab35d3d)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:35.188741 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ad8a5-73e6-4d0e-b4c6-582a63fe617d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:35.190375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 606/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:04:35 2022] GET /identity/v3/users/2fcfe533d3534423905f9121aab35d3d => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:35.196028 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:35.290510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b8730-d1fe-4f69-b5bc-b80b4322b4b1 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:35.321215 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b515bf-8467-433c-aa61-3191a4a04392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:35.322053 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b515bf-8467-433c-aa61-3191a4a04392 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:35.322335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b515bf-8467-433c-aa61-3191a4a04392 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:35.322605 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b515bf-8467-433c-aa61-3191a4a04392 None admin] PATH_INFO: `/v3/projects/2b40a75d89564271826427e6e9d932a5` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:35.323763 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b515bf-8467-433c-aa61-3191a4a04392 None admin] RBAC: Authorizing `identity:get_project(project_id=2b40a75d89564271826427e6e9d932a5)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:35.325212 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b515bf-8467-433c-aa61-3191a4a04392 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:35.326895 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 585/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:04:35 2022] GET /identity/v3/projects/2b40a75d89564271826427e6e9d932a5 => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:35.332668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad8a5-73e6-4d0e-b4c6-582a63fe617d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:35.429952 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad8a5-73e6-4d0e-b4c6-582a63fe617d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:35.461433 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3600-3a21-4b16-9d5b-824ff9a0df20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:35.462252 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3600-3a21-4b16-9d5b-824ff9a0df20 None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:35.462532 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3600-3a21-4b16-9d5b-824ff9a0df20 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:35.462804 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3600-3a21-4b16-9d5b-824ff9a0df20 None admin] PATH_INFO: `/v3/projects/2b40a75d89564271826427e6e9d932a5/users/2fcfe533d3534423905f9121aab35d3d/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:35.464730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c3600-3a21-4b16-9d5b-824ff9a0df20 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b40a75d89564271826427e6e9d932a5, user_id=2fcfe533d3534423905f9121aab35d3d, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:35.466198 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c3600-3a21-4b16-9d5b-824ff9a0df20 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:35.478740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 607/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:04:35 2022] PUT /identity/v3/projects/2b40a75d89564271826427e6e9d932a5/users/2fcfe533d3534423905f9121aab35d3d/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:04:37.669388 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55312bdf-b5fa-4295-9893-1080923f143f None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:37.669917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55312bdf-b5fa-4295-9893-1080923f143f None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:37.669917 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55312bdf-b5fa-4295-9893-1080923f143f None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:37.671296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 586/1193] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:04:37.676337 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0e97-8a3b-4a84-b47b-297cd7100228 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:37.676730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0e97-8a3b-4a84-b47b-297cd7100228 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:37.676886 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0e97-8a3b-4a84-b47b-297cd7100228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:37.694035 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-61fc0e97-8a3b-4a84-b47b-297cd7100228 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:04:37.695273 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-61fc0e97-8a3b-4a84-b47b-297cd7100228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:04:37.757672 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 608/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:04:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:04:37.780643 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55312bdf-b5fa-4295-9893-1080923f143f None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:37.875745 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55312bdf-b5fa-4295-9893-1080923f143f None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:37.907233 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634753d2-519c-4122-8f00-943f68b75672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:37.907993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634753d2-519c-4122-8f00-943f68b75672 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:37.908270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634753d2-519c-4122-8f00-943f68b75672 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:37.908542 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634753d2-519c-4122-8f00-943f68b75672 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:37.918449 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634753d2-519c-4122-8f00-943f68b75672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-634753d2-519c-4122-8f00-943f68b75672 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:04:37.926010 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:04:37.927092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:04:37.928143 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 587/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:04:37.932817 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc0e97-8a3b-4a84-b47b-297cd7100228 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:38.026138 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc0e97-8a3b-4a84-b47b-297cd7100228 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:38.056793 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:38.057470 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:38.057747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:38.058014 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:38.058748 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:38.059047 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:38.060389 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:38.067868 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 609/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:38.074169 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634753d2-519c-4122-8f00-943f68b75672 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:38.167574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634753d2-519c-4122-8f00-943f68b75672 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:38.198980 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49821e-46f3-4596-b7d1-896ca3440c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:38.199779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49821e-46f3-4596-b7d1-896ca3440c84 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:38.200057 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49821e-46f3-4596-b7d1-896ca3440c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:38.200327 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49821e-46f3-4596-b7d1-896ca3440c84 None admin] PATH_INFO: `/v3/users/2fcfe533d3534423905f9121aab35d3d` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:38.201509 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49821e-46f3-4596-b7d1-896ca3440c84 None admin] RBAC: Authorizing `identity:get_user(user_id=2fcfe533d3534423905f9121aab35d3d)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:38.202941 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49821e-46f3-4596-b7d1-896ca3440c84 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:38.204554 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 588/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:04:38 2022] GET /identity/v3/users/2fcfe533d3534423905f9121aab35d3d => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:38.210768 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:38.303733 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4a48e-b7d8-4667-89af-6fa0255748b6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:38.333849 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae740b-f05c-4e61-9a2f-12e02b55a406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:38.334467 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae740b-f05c-4e61-9a2f-12e02b55a406 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:38.334742 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae740b-f05c-4e61-9a2f-12e02b55a406 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:38.335010 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae740b-f05c-4e61-9a2f-12e02b55a406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:38.396896 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae740b-f05c-4e61-9a2f-12e02b55a406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:38.398867 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae740b-f05c-4e61-9a2f-12e02b55a406 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:38.488517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 610/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:04:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:04:38.494192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49821e-46f3-4596-b7d1-896ca3440c84 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:38.585987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49821e-46f3-4596-b7d1-896ca3440c84 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:38.617595 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b04e99-4388-4734-a99f-956d29721473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:38.618427 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b04e99-4388-4734-a99f-956d29721473 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:38.618704 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b04e99-4388-4734-a99f-956d29721473 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:38.618975 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b04e99-4388-4734-a99f-956d29721473 None admin] PATH_INFO: `/v3/projects/2b40a75d89564271826427e6e9d932a5` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:38.620134 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b04e99-4388-4734-a99f-956d29721473 None admin] RBAC: Authorizing `identity:get_project(project_id=2b40a75d89564271826427e6e9d932a5)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:38.621582 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b04e99-4388-4734-a99f-956d29721473 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:38.623247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 589/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:04:38 2022] GET /identity/v3/projects/2b40a75d89564271826427e6e9d932a5 => generated 361 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:04:38.629282 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae740b-f05c-4e61-9a2f-12e02b55a406 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:04:38.721436 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae740b-f05c-4e61-9a2f-12e02b55a406 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:04:38.752924 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c5d77-f121-4a26-8e09-9777690455d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:04:38.753531 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c5d77-f121-4a26-8e09-9777690455d1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:04:38.753814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c5d77-f121-4a26-8e09-9777690455d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:04:38.754088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c5d77-f121-4a26-8e09-9777690455d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:04:38.754916 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c5d77-f121-4a26-8e09-9777690455d1 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=2b40a75d89564271826427e6e9d932a5, user.id=2fcfe533d3534423905f9121aab35d3d) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:04:38.755228 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c5d77-f121-4a26-8e09-9777690455d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:04:38.756633 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c5d77-f121-4a26-8e09-9777690455d1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:04:38.762744 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 611/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:04:38 2022] GET /identity/v3/role_assignments?user.id=2fcfe533d3534423905f9121aab35d3d&scope.project.id=2b40a75d89564271826427e6e9d932a5&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:39.519512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b254-405b-431b-896d-88f75aee5f34 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:39.520101 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b254-405b-431b-896d-88f75aee5f34 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:39.520101 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7b254-405b-431b-896d-88f75aee5f34 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:39.521972 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 590/1201] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:05:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:05:39.527472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f525dc-53e4-4f83-9b37-1c7c5f9dd8b5 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:39.527953 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f525dc-53e4-4f83-9b37-1c7c5f9dd8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:39.528034 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f525dc-53e4-4f83-9b37-1c7c5f9dd8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:39.547740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-20f525dc-53e4-4f83-9b37-1c7c5f9dd8b5 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:05:39.549048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-20f525dc-53e4-4f83-9b37-1c7c5f9dd8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:05:39.634030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 612/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:05:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:05:39.663655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b254-405b-431b-896d-88f75aee5f34 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:39.783665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7b254-405b-431b-896d-88f75aee5f34 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:39.824270 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:39.825365 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:39.825648 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:39.825931 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:39.836594 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:39.838063 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:39.858549 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:39.859697 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:39.860840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:39.860840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:39.860840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:39.860840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:39.865609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 591/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:39.866912 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f525dc-53e4-4f83-9b37-1c7c5f9dd8b5 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:39.983776 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f525dc-53e4-4f83-9b37-1c7c5f9dd8b5 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:40.025358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10534a-4c97-4106-99b4-48b98e560433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:40.026023 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10534a-4c97-4106-99b4-48b98e560433 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:40.026316 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10534a-4c97-4106-99b4-48b98e560433 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:40.026575 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10534a-4c97-4106-99b4-48b98e560433 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:40.027296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10534a-4c97-4106-99b4-48b98e560433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:40.027598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10534a-4c97-4106-99b4-48b98e560433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:40.029030 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10534a-4c97-4106-99b4-48b98e560433 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:40.042093 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 613/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:40.048229 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:40.168650 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2102f-6480-453b-897c-bb29ad6068ac None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:40.206609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66f42f-fd1b-47c7-aa80-8ba9b0b92005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:40.207292 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f42f-fd1b-47c7-aa80-8ba9b0b92005 None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:40.207564 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f42f-fd1b-47c7-aa80-8ba9b0b92005 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:40.207865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f42f-fd1b-47c7-aa80-8ba9b0b92005 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:40.208567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66f42f-fd1b-47c7-aa80-8ba9b0b92005 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:40.210324 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66f42f-fd1b-47c7-aa80-8ba9b0b92005 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:40.232696 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 592/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:05:40 2022] POST /identity/v3/users => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:05:42.590963 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd627f29-e660-4b56-9ffd-39e805eed94f None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:42.591522 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd627f29-e660-4b56-9ffd-39e805eed94f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:42.591522 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd627f29-e660-4b56-9ffd-39e805eed94f None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:42.592865 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 614/1206] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:05:42.598298 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140b39-d0e1-4872-9695-2ec287afd1b8 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:42.598765 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140b39-d0e1-4872-9695-2ec287afd1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:42.598865 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140b39-d0e1-4872-9695-2ec287afd1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:42.615822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-07140b39-d0e1-4872-9695-2ec287afd1b8 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:05:42.617011 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-07140b39-d0e1-4872-9695-2ec287afd1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:05:42.693420 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 593/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:05:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:05:42.719619 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd627f29-e660-4b56-9ffd-39e805eed94f None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:42.837854 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd627f29-e660-4b56-9ffd-39e805eed94f None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:42.876403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b9833-860b-4dda-a466-8ffbc90193e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:42.877369 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b9833-860b-4dda-a466-8ffbc90193e4 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:42.877650 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b9833-860b-4dda-a466-8ffbc90193e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:42.877920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b9833-860b-4dda-a466-8ffbc90193e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:42.891028 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b9833-860b-4dda-a466-8ffbc90193e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-fc2b9833-860b-4dda-a466-8ffbc90193e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:05:42.899731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:42.900756 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:42.902039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 615/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:05:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:42.907394 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07140b39-d0e1-4872-9695-2ec287afd1b8 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:43.023552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07140b39-d0e1-4872-9695-2ec287afd1b8 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:43.063765 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:43.064446 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:43.064726 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:43.064998 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:43.065787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:43.066094 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:43.068046 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:43.076467 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 594/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:05:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:43.082628 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b9833-860b-4dda-a466-8ffbc90193e4 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:43.202968 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b9833-860b-4dda-a466-8ffbc90193e4 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:43.246376 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:43.247500 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:43.247735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:43.247994 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:43.259216 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:43.260658 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:43.279709 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:43.280674 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:43.287044 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:43.287044 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:43.287044 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:43.287044 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:43.287044 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 616/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:05:43.287411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:43.396804 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eccf2-6ebf-47af-8432-a1b8966a2b11 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:43.433698 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:43.434411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:43.434688 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:43.434954 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:43.435800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:43.436101 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:43.437811 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:43.456732 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 595/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:43.458304 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:43.570811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e143f-6530-4a7e-bbca-89ed23f0b3d6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:43.608869 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:43.610712 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:43.611181 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:43.611676 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:43.622601 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:43.624549 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:05:43.643883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:43.645297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:05:43.646277 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:43.650404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 617/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:43.651244 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:43.766085 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e996aad-659a-439a-aea6-3bee3f74e944 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:43.800714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:43.802346 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:43.802858 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:43.803415 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:43.804512 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:43.804920 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:43.806463 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:43.833534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 596/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:05:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:05:43.835241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:43.939719 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a098882-e1fe-4e49-85a1-41a831172ae9 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:43.977481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf02a8-ba9e-4952-965a-ea2ac75b48f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:43.978150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf02a8-ba9e-4952-965a-ea2ac75b48f2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:43.978426 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf02a8-ba9e-4952-965a-ea2ac75b48f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:43.978694 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf02a8-ba9e-4952-965a-ea2ac75b48f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:44.055741 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf02a8-ba9e-4952-965a-ea2ac75b48f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:44.057349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf02a8-ba9e-4952-965a-ea2ac75b48f2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:44.178758 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 618/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:05:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:05:44.181482 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:44.295904 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbed036-ae51-458e-bebc-098184ebbdfd None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:44.337328 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:44.338253 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:44.338534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:44.338800 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:44.350837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:44.352617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:44.375839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:44.376810 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:44.382510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:44.382510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:44.382510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:44.382510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:44.382510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 597/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:44.382898 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf02a8-ba9e-4952-965a-ea2ac75b48f2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:44.497393 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf02a8-ba9e-4952-965a-ea2ac75b48f2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:44.541079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:44.542336 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:44.542622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:44.542900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:44.543804 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:44.544230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:44.546026 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:44.564573 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 619/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:44.566016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:44.685757 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0ac7f-b13b-4b42-9c4a-edfad5e0b0c7 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:44.724838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:44.725675 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:44.725952 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:44.726224 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:44.735274 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:44.737241 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:05:44.754462 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:44.755592 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:44.758138 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 598/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:05:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:44.761375 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:44.887751 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f973d7-1990-442d-bf81-f256f8ee2bf6 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:44.924518 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ecc36-52a7-435a-acb5-1eadbcbe01f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:44.925226 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ecc36-52a7-435a-acb5-1eadbcbe01f3 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:44.925475 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ecc36-52a7-435a-acb5-1eadbcbe01f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:44.925745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ecc36-52a7-435a-acb5-1eadbcbe01f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:44.927132 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ecc36-52a7-435a-acb5-1eadbcbe01f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:44.927442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ecc36-52a7-435a-acb5-1eadbcbe01f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:44.928925 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ecc36-52a7-435a-acb5-1eadbcbe01f3 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:44.943187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 620/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:05:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:05:44.949180 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:45.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5524-71b1-4c0e-a466-e4bce3f34d71 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:45.093602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a861069-c64f-4ba1-a7fe-0818adc82274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:45.094231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a861069-c64f-4ba1-a7fe-0818adc82274 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:45.094507 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a861069-c64f-4ba1-a7fe-0818adc82274 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:45.094773 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a861069-c64f-4ba1-a7fe-0818adc82274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:45.095602 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a861069-c64f-4ba1-a7fe-0818adc82274 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=f32b7537dc534f9da356c4230e27ef8e) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:45.095926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a861069-c64f-4ba1-a7fe-0818adc82274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:45.097365 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a861069-c64f-4ba1-a7fe-0818adc82274 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:45.103657 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 599/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:05:44 2022] GET /identity/v3/role_assignments?user.id=f32b7537dc534f9da356c4230e27ef8e&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:05:47.362380 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ff1bf-704f-4fc7-9e51-400ecafa5072 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:47.362947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ff1bf-704f-4fc7-9e51-400ecafa5072 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:47.362947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ff1bf-704f-4fc7-9e51-400ecafa5072 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:47.364584 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 621/1220] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:05:47.369680 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef8fe4-8163-489c-9053-69c325736435 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:47.370149 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef8fe4-8163-489c-9053-69c325736435 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:47.370230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef8fe4-8163-489c-9053-69c325736435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:47.387218 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-57ef8fe4-8163-489c-9053-69c325736435 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:05:47.388320 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-57ef8fe4-8163-489c-9053-69c325736435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:05:47.460546 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 600/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:05:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:05:47.485746 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ff1bf-704f-4fc7-9e51-400ecafa5072 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:47.613009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ff1bf-704f-4fc7-9e51-400ecafa5072 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:47.655296 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6508973-c1c8-4726-b5a8-361a67a805ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:47.656013 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6508973-c1c8-4726-b5a8-361a67a805ca None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:47.656376 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6508973-c1c8-4726-b5a8-361a67a805ca None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:47.656582 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6508973-c1c8-4726-b5a8-361a67a805ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:47.669415 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6508973-c1c8-4726-b5a8-361a67a805ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-a6508973-c1c8-4726-b5a8-361a67a805ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:05:47.678759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:47.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:47.681124 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 622/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:05:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:05:47.687074 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef8fe4-8163-489c-9053-69c325736435 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:47.798969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef8fe4-8163-489c-9053-69c325736435 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:47.835479 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:47.836147 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:47.836416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:47.836716 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:47.837653 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] RBAC: Adding query filter params (name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:47.837960 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:47.839692 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:47.847674 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 601/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:05:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:47.854396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6508973-c1c8-4726-b5a8-361a67a805ca None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:47.958395 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6508973-c1c8-4726-b5a8-361a67a805ca None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:47.993092 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:47.993876 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:47.994152 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:47.994421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:48.005896 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:48.007478 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:48.024368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:48.025441 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:48.026542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:48.026542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:48.026542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:48.026542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:48.026542 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 623/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:48.031808 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:48.137791 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0ea67-32a3-46cc-affd-b1bd18f5f315 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:48.176209 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:48.176875 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:48.177187 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:48.177464 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:48.178190 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:48.178556 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:48.179839 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:48.192005 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 602/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:05:48.197999 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:48.312938 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7860-ce11-4d65-b2b7-286436fc6cc5 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:48.351224 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:48.352088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:48.352367 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:48.352688 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:48.360785 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:48.362418 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:05:48.378745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:48.381146 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:48.382243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 624/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:48.387183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:48.490488 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1509b6-60fd-439a-8702-1b0f44064241 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:48.528373 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:48.529052 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:48.529339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:48.529609 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:48.530438 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:48.530741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:48.532192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:48.550517 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 603/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:05:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:48.556168 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:48.657739 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881d3c9-0888-47c9-9564-217b3001fd98 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:48.694160 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:48.695020 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:48.695311 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:48.695567 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:48.706349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:48.708353 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:48.725404 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:48.726745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:48.728007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:48.728007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:48.728007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:48.728007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:48.728007 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 625/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:48.732291 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:48.836369 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6ae0-bcb3-4585-9944-c68a993a1bac None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:48.873465 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:48.874104 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:48.874417 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:48.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:48.875742 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:48.876043 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:48.878787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:48.891065 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 604/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:48.896886 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:49.002895 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a84c1-6540-4ad8-b3f2-8976a1bd7db2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:49.038174 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:49.039183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:49.039556 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:49.039833 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:49.047062 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:49.048495 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:05:49.063481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:49.064580 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:49.065722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 626/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:49.071352 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:49.176852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eeb13-aa35-4e21-87fb-2a4b4c66b279 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:49.209591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62947a1c-2085-4b66-bbb7-27359dea9913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:49.210339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62947a1c-2085-4b66-bbb7-27359dea9913 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:49.210638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62947a1c-2085-4b66-bbb7-27359dea9913 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:49.210950 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62947a1c-2085-4b66-bbb7-27359dea9913 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:49.211819 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62947a1c-2085-4b66-bbb7-27359dea9913 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:49.212129 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62947a1c-2085-4b66-bbb7-27359dea9913 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:49.213718 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62947a1c-2085-4b66-bbb7-27359dea9913 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:49.229567 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 605/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:05:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:05:49.236136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:49.341823 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e40728-e291-4576-9989-7186a6fb0945 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:49.377181 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3922fe-925e-44af-8f63-fbedbaeeddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:49.378062 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3922fe-925e-44af-8f63-fbedbaeeddce None admin] REQUEST_METHOD: `PUT` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:49.378335 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3922fe-925e-44af-8f63-fbedbaeeddce None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:49.378599 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3922fe-925e-44af-8f63-fbedbaeeddce None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/f32b7537dc534f9da356c4230e27ef8e/roles/48c24d23f41249abbb5224c3fa457f9e` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:49.393177 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3922fe-925e-44af-8f63-fbedbaeeddce None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=f32b7537dc534f9da356c4230e27ef8e, role_id=48c24d23f41249abbb5224c3fa457f9e)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:49.394720 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3922fe-925e-44af-8f63-fbedbaeeddce None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:49.410853 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 627/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:05:49 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/f32b7537dc534f9da356c4230e27ef8e/roles/48c24d23f41249abbb5224c3fa457f9e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:05:51.725624 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61975bfe-24d7-4a7f-a4c2-a97f708cd4f9 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:51.726314 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61975bfe-24d7-4a7f-a4c2-a97f708cd4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:51.726426 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61975bfe-24d7-4a7f-a4c2-a97f708cd4f9 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:51.728096 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 606/1233] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:05:51.734034 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef9cb-05b7-4f33-9c37-53704f0241b6 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:51.734621 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef9cb-05b7-4f33-9c37-53704f0241b6 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:51.734621 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ef9cb-05b7-4f33-9c37-53704f0241b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:51.752863 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-930ef9cb-05b7-4f33-9c37-53704f0241b6 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:05:51.754148 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-930ef9cb-05b7-4f33-9c37-53704f0241b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:05:51.839063 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 628/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:05:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:05:51.867397 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61975bfe-24d7-4a7f-a4c2-a97f708cd4f9 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:51.981455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61975bfe-24d7-4a7f-a4c2-a97f708cd4f9 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:52.016347 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560c15c-d0b6-4d0e-8017-16de69f27f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:52.017161 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560c15c-d0b6-4d0e-8017-16de69f27f0a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:52.017457 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560c15c-d0b6-4d0e-8017-16de69f27f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:52.017714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560c15c-d0b6-4d0e-8017-16de69f27f0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:52.028862 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560c15c-d0b6-4d0e-8017-16de69f27f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-0560c15c-d0b6-4d0e-8017-16de69f27f0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:05:52.037214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:52.038383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:52.039265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 607/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:05:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:05:52.048429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ef9cb-05b7-4f33-9c37-53704f0241b6 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:52.162598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ef9cb-05b7-4f33-9c37-53704f0241b6 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:52.206950 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:52.207657 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:52.207939 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:52.208213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:52.208970 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] RBAC: Adding query filter params (name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:52.209308 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:52.210677 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:52.220042 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 629/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:52.226838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560c15c-d0b6-4d0e-8017-16de69f27f0a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:52.334185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560c15c-d0b6-4d0e-8017-16de69f27f0a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:52.370332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:52.371196 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:52.371475 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:52.371747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:52.382656 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:52.384106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:52.402121 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:52.403499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:52.404639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:52.404639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:52.404639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:52.404639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:52.404639 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 608/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:05:52.408961 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:52.509898 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22eab6-6045-419d-90c1-f22f0fa0473d None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:52.545613 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:52.546415 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:52.546723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:52.547043 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:52.547846 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:52.548229 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:52.549710 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:52.563344 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 630/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:52.569424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:52.675211 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53eb02-9000-4dbd-b9cd-7a77e767eea5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:52.711226 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:52.712021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:52.712300 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:52.712573 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:52.721731 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:52.724175 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:05:52.739398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:52.741491 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:52.743108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 609/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:05:52.747488 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:52.856977 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c86c00-c367-4ab4-a948-c7f97fa6f6d3 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:52.891833 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82739801-8761-40e9-a094-f3651695874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:52.892585 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82739801-8761-40e9-a094-f3651695874c None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:52.892868 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82739801-8761-40e9-a094-f3651695874c None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:52.893172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82739801-8761-40e9-a094-f3651695874c None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:52.895763 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82739801-8761-40e9-a094-f3651695874c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:52.896099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82739801-8761-40e9-a094-f3651695874c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:52.897545 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82739801-8761-40e9-a094-f3651695874c None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:52.915577 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 631/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:05:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:05:52.921354 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:53.025089 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68090d1d-516b-4529-af77-241ad581d39d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:53.058608 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb078-5818-46c5-8653-117e848ccae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:53.059404 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb078-5818-46c5-8653-117e848ccae5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:53.059683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb078-5818-46c5-8653-117e848ccae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:53.059954 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bb078-5818-46c5-8653-117e848ccae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:53.132001 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bb078-5818-46c5-8653-117e848ccae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:53.133510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bb078-5818-46c5-8653-117e848ccae5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:53.233158 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 610/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:05:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:05:53.240355 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82739801-8761-40e9-a094-f3651695874c None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:53.346879 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82739801-8761-40e9-a094-f3651695874c None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:53.382443 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:53.383236 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:53.383529 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:53.383810 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:53.393735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:53.395785 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:53.414019 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:53.415358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:53.416661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:53.416661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:53.416661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:53.416661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:53.416661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 632/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:53.420910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb078-5818-46c5-8653-117e848ccae5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:53.521130 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bb078-5818-46c5-8653-117e848ccae5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:53.556145 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:53.556801 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:53.558411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:53.558411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:53.558411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:53.558411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:53.560663 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:53.573894 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 611/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:05:53.579414 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:53.683421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55091535-7ef0-49a1-a290-94b9e2b040e2 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:53.720056 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:53.720900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:53.721209 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:53.721479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:53.727997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:53.729695 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:05:53.745464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:53.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:53.748074 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 633/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:53.753088 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:53.859073 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1d431-c4e4-4280-8854-9a5eb2a544a5 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:53.898931 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9383edc-9679-42be-a9f8-6c8b75c53fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:53.899629 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383edc-9679-42be-a9f8-6c8b75c53fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:53.899910 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383edc-9679-42be-a9f8-6c8b75c53fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:53.900183 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9383edc-9679-42be-a9f8-6c8b75c53fc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:53.900972 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9383edc-9679-42be-a9f8-6c8b75c53fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:53.901315 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9383edc-9679-42be-a9f8-6c8b75c53fc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:53.902761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9383edc-9679-42be-a9f8-6c8b75c53fc1 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:53.916192 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 612/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:53.922161 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:54.026291 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b82f-7e5f-4ede-9b1d-523b769a5d36 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:54.059799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81066ba-9ad9-45db-afff-a40cfeaee743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:54.060448 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81066ba-9ad9-45db-afff-a40cfeaee743 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:54.060732 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81066ba-9ad9-45db-afff-a40cfeaee743 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:54.061009 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81066ba-9ad9-45db-afff-a40cfeaee743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:54.061900 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81066ba-9ad9-45db-afff-a40cfeaee743 None admin] RBAC: Adding query filter params (role.id=48c24d23f41249abbb5224c3fa457f9e, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=f32b7537dc534f9da356c4230e27ef8e) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:54.062214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81066ba-9ad9-45db-afff-a40cfeaee743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:54.063634 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81066ba-9ad9-45db-afff-a40cfeaee743 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:54.070055 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 634/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:05:53 2022] GET /identity/v3/role_assignments?user.id=f32b7537dc534f9da356c4230e27ef8e&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=48c24d23f41249abbb5224c3fa457f9e => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:56.398836 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d912f-4fd6-4c55-86f5-fecc093656d6 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:56.399455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d912f-4fd6-4c55-86f5-fecc093656d6 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:56.399455 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d912f-4fd6-4c55-86f5-fecc093656d6 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:56.400847 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 613/1247] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:05:56.406755 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad040a4-cfd4-4b36-9d52-7872e5f4757f None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:56.407293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad040a4-cfd4-4b36-9d52-7872e5f4757f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:56.407293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad040a4-cfd4-4b36-9d52-7872e5f4757f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:56.424629 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-8ad040a4-cfd4-4b36-9d52-7872e5f4757f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:05:56.426429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-8ad040a4-cfd4-4b36-9d52-7872e5f4757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:05:56.498374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 635/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:05:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:05:56.525067 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d912f-4fd6-4c55-86f5-fecc093656d6 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:56.640880 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d912f-4fd6-4c55-86f5-fecc093656d6 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:56.676321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968775e7-9bfe-4a8f-b494-c1733442ca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:56.677021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968775e7-9bfe-4a8f-b494-c1733442ca92 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:56.677357 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968775e7-9bfe-4a8f-b494-c1733442ca92 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:56.677659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968775e7-9bfe-4a8f-b494-c1733442ca92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:56.689170 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968775e7-9bfe-4a8f-b494-c1733442ca92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-968775e7-9bfe-4a8f-b494-c1733442ca92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:05:56.697877 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:56.699040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:56.700193 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 614/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:05:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:56.706170 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad040a4-cfd4-4b36-9d52-7872e5f4757f None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:56.812715 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad040a4-cfd4-4b36-9d52-7872e5f4757f None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:56.848336 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:56.849329 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:56.849609 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:56.849881 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:56.850723 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:56.851126 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:56.852628 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:56.861873 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 636/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:05:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:56.868670 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968775e7-9bfe-4a8f-b494-c1733442ca92 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:56.978908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968775e7-9bfe-4a8f-b494-c1733442ca92 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:57.014834 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:57.015749 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:57.015992 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:57.016263 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:57.027838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:57.029325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:57.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:57.050349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:57.051526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:57.051526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:57.051526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:57.051526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:57.051526 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 615/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:57.057437 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:57.166583 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b9ea9-48d5-44fb-b469-651efe1bf96a None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:57.200510 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:57.201198 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:57.201477 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:57.201747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:57.202474 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:57.202779 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:57.204159 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:57.218207 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 637/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:57.225176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:57.331137 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858f01-f690-4d12-9443-891e4bee647a None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:57.365809 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:57.366623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:57.366907 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:57.367184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:57.375647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:57.377144 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:05:57.392655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:57.393911 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:05:57.395002 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 616/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:57.400411 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:57.500085 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a4316-5bda-4586-a4e2-2934aff39526 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:57.533622 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:57.534283 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:57.534554 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:57.534821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:57.535648 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:57.535953 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:57.537398 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:57.554202 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 638/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:05:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:57.559873 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:57.659030 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea69d-cec3-426d-92b8-c20a8d23f7e3 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:57.691873 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d045f-496a-4d74-93ea-eed8ce6567ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:57.692518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d045f-496a-4d74-93ea-eed8ce6567ca None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:57.692849 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d045f-496a-4d74-93ea-eed8ce6567ca None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:57.693154 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d045f-496a-4d74-93ea-eed8ce6567ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:57.757501 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d045f-496a-4d74-93ea-eed8ce6567ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:57.758949 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d045f-496a-4d74-93ea-eed8ce6567ca None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:57.851679 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 617/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:05:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:05:57.859299 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:57.952745 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d997de0-e257-487b-a9a0-b52560820ab1 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:57.983821 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:57.984589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:57.984870 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:57.985172 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:57.995127 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:57.996576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:05:58.014070 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:58.015101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:58.016196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:05:58.016196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:05:58.016196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:05:58.016196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:58.016196 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 639/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:58.020694 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d045f-496a-4d74-93ea-eed8ce6567ca None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:58.111340 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d045f-496a-4d74-93ea-eed8ce6567ca None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:58.144100 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:58.144781 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:58.145087 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:58.145364 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:58.146106 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:58.146411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:58.147761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:58.158372 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 618/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:58.164338 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:58.263877 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e436d42-32fc-45f4-9bad-abc2bc92453f None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:58.299472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:58.300248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:58.300531 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:58.301431 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:58.307955 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:58.309422 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:05:58.322724 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:05:58.323735 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:05:58.324701 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 640/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:05:58.329736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:58.423331 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ba21e-0cf5-46f8-9268-1c84a819e5a7 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:58.454437 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c99138-d283-4afe-8890-317344d175eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:58.455068 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99138-d283-4afe-8890-317344d175eb None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:58.455341 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99138-d283-4afe-8890-317344d175eb None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:58.455606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c99138-d283-4afe-8890-317344d175eb None admin] PATH_INFO: `/v3/projects` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:58.456380 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c99138-d283-4afe-8890-317344d175eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:58.456690 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c99138-d283-4afe-8890-317344d175eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:58.458123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c99138-d283-4afe-8890-317344d175eb None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:58.469296 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 619/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:05:58.475771 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:05:58.569309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbc75-c987-461e-b493-54b2bc86820e None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:05:58.601128 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d925a6-ab6b-4de2-8f6d-27f3844af7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:05:58.601761 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d925a6-ab6b-4de2-8f6d-27f3844af7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:05:58.602039 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d925a6-ab6b-4de2-8f6d-27f3844af7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:05:58.602308 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d925a6-ab6b-4de2-8f6d-27f3844af7b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:05:58.603131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d925a6-ab6b-4de2-8f6d-27f3844af7b1 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=f32b7537dc534f9da356c4230e27ef8e) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:05:58.603435 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d925a6-ab6b-4de2-8f6d-27f3844af7b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:05:58.604846 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d925a6-ab6b-4de2-8f6d-27f3844af7b1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:05:58.610191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 641/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:05:58 2022] GET /identity/v3/role_assignments?user.id=f32b7537dc534f9da356c4230e27ef8e&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:00.783383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335f401-b3b4-4950-96fe-619f22fe5b34 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:00.784016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335f401-b3b4-4950-96fe-619f22fe5b34 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:00.784016 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335f401-b3b4-4950-96fe-619f22fe5b34 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:00.785908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 620/1261] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:06:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:06:00.790984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb945-cf19-4e0e-b2de-95342578aea0 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:00.791399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb945-cf19-4e0e-b2de-95342578aea0 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:00.791516 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb945-cf19-4e0e-b2de-95342578aea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:00.807799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-eaecb945-cf19-4e0e-b2de-95342578aea0 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:06:00.808973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-eaecb945-cf19-4e0e-b2de-95342578aea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:06:00.871946 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 642/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:06:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:06:00.894058 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335f401-b3b4-4950-96fe-619f22fe5b34 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:00.988638 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335f401-b3b4-4950-96fe-619f22fe5b34 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:01.020347 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee83d6-3f02-4ca2-b1a1-bd75b4dfdc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:01.021131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee83d6-3f02-4ca2-b1a1-bd75b4dfdc26 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:01.021410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee83d6-3f02-4ca2-b1a1-bd75b4dfdc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:01.021678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee83d6-3f02-4ca2-b1a1-bd75b4dfdc26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:01.031148 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee83d6-3f02-4ca2-b1a1-bd75b4dfdc26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-66ee83d6-3f02-4ca2-b1a1-bd75b4dfdc26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:06:01.039257 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:01.040386 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:06:01.041695 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 621/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:06:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:01.046202 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb945-cf19-4e0e-b2de-95342578aea0 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:01.138578 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb945-cf19-4e0e-b2de-95342578aea0 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:01.170135 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:01.170780 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:01.171059 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:01.171330 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] PATH_INFO: `/v3/roles` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:01.172076 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:01.172381 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:01.173771 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:01.181430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 643/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:06:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:01.187023 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee83d6-3f02-4ca2-b1a1-bd75b4dfdc26 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:01.292361 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee83d6-3f02-4ca2-b1a1-bd75b4dfdc26 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:01.325214 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:01.325982 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:01.326259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:01.326529 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:01.336913 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:01.338383 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:06:01.354440 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:01.355454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:01.356407 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:06:01.356407 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:06:01.356407 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:06:01.356407 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:06:01.356407 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 622/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:01.361303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:01.454498 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c8718-f5f5-4442-b3ab-03d1e28505fd None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:01.485464 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:01.486101 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:01.486380 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:01.486652 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:01.487358 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:01.487661 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:01.489069 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:01.500303 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 644/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:01.506531 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:01.601568 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60ac2a-b4e8-4b8b-8742-5d391d17c372 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:01.632038 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:01.632815 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:01.633122 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:01.633390 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:01.640308 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:01.641745 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:06:01.655835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:01.656926 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:06:01.658230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 623/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:06:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:01.664788 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:01.757130 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0915a3d-b74b-46e0-a97a-78ccf2f19352 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:01.788726 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:01.789390 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:01.789667 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:01.789937 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] PATH_INFO: `/v3/users` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:01.790754 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:01.791056 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:01.792455 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:01.808324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 645/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:06:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:01.813848 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:01.906978 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5200db-bf0f-414e-a868-5d476359fd7f None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:01.937022 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:01.937803 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:01.938083 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:01.938350 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:01.948613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:01.950056 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:06:01.966601 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:01.967665 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:01.973486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:06:01.973486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:06:01.973486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:06:01.973486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:06:01.973486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 624/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:06:01.973840 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:02.067139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c94b5-54e1-48b1-973a-85582771dd7b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:02.098719 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:02.099356 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:02.099636 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:02.099908 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:02.100620 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:02.100920 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:02.102308 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:02.116477 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 646/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:02.119278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:02.212312 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c129d6-ae8d-41a5-ac99-9b7c9005b8c4 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:02.247387 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:02.248176 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:02.248450 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:02.248722 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:02.255869 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:02.257600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:06:02.272310 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:02.273398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:06:02.274432 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 625/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:02.279276 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:02.373157 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c95bf-7656-4e09-8d09-01ab9b209dcd None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:02.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661be19-60ac-45d1-aa2c-9b05d5b15726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:02.405010 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661be19-60ac-45d1-aa2c-9b05d5b15726 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:02.405318 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661be19-60ac-45d1-aa2c-9b05d5b15726 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:02.405589 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661be19-60ac-45d1-aa2c-9b05d5b15726 None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:02.406363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661be19-60ac-45d1-aa2c-9b05d5b15726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:02.406668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661be19-60ac-45d1-aa2c-9b05d5b15726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:02.408079 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661be19-60ac-45d1-aa2c-9b05d5b15726 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:02.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 647/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:02.425783 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:02.520103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e07c3-51ba-4c25-918a-0728ee812192 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:02.553798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb994b-9d92-4843-bd11-a146f2176dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:02.554677 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb994b-9d92-4843-bd11-a146f2176dac None admin] REQUEST_METHOD: `PUT` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:02.554958 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb994b-9d92-4843-bd11-a146f2176dac None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:02.555231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb994b-9d92-4843-bd11-a146f2176dac None admin] PATH_INFO: `/v3/projects/46e5f537a906431ba5baa049a20664df/users/f32b7537dc534f9da356c4230e27ef8e/roles/c73ae1fee2a2491482532db2e578cc32` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:02.557185 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb994b-9d92-4843-bd11-a146f2176dac None admin] RBAC: Authorizing `identity:create_grant(project_id=46e5f537a906431ba5baa049a20664df, user_id=f32b7537dc534f9da356c4230e27ef8e, role_id=c73ae1fee2a2491482532db2e578cc32)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:02.558606 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb994b-9d92-4843-bd11-a146f2176dac None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:02.570807 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 626/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:06:02 2022] PUT /identity/v3/projects/46e5f537a906431ba5baa049a20664df/users/f32b7537dc534f9da356c4230e27ef8e/roles/c73ae1fee2a2491482532db2e578cc32 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:06:04.730912 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b49bf8-4b57-4afa-b17c-7c64107fada4 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:04.731442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b49bf8-4b57-4afa-b17c-7c64107fada4 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:04.731442 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b49bf8-4b57-4afa-b17c-7c64107fada4 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:04.732819 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 648/1274] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:06:04.738194 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bd9b3-d233-487d-a7c2-a9c5e5bbabf1 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:04.738633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bd9b3-d233-487d-a7c2-a9c5e5bbabf1 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:04.738744 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bd9b3-d233-487d-a7c2-a9c5e5bbabf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:04.755079 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-0d4bd9b3-d233-487d-a7c2-a9c5e5bbabf1 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:06:04.756259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-0d4bd9b3-d233-487d-a7c2-a9c5e5bbabf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:06:04.819525 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 627/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:06:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:06:04.842697 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b49bf8-4b57-4afa-b17c-7c64107fada4 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:04.938571 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b49bf8-4b57-4afa-b17c-7c64107fada4 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:04.970507 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9213160-cc02-4b55-bfd2-084bc73da0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:04.971269 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9213160-cc02-4b55-bfd2-084bc73da0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:04.971547 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9213160-cc02-4b55-bfd2-084bc73da0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:04.971818 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9213160-cc02-4b55-bfd2-084bc73da0f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:04.981590 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9213160-cc02-4b55-bfd2-084bc73da0f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-f9213160-cc02-4b55-bfd2-084bc73da0f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:06:04.989176 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:04.990300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:04.991480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 649/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:06:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:04.996226 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bd9b3-d233-487d-a7c2-a9c5e5bbabf1 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:05.088613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bd9b3-d233-487d-a7c2-a9c5e5bbabf1 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:05.120715 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:05.121410 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:05.121694 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:05.121977 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] PATH_INFO: `/v3/roles` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:05.122734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:05.123040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:05.124979 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:05.132996 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 628/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:06:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:05.138748 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9213160-cc02-4b55-bfd2-084bc73da0f1 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:05.235976 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9213160-cc02-4b55-bfd2-084bc73da0f1 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:05.275052 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:05.275949 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:05.276281 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:05.276607 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:05.289376 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:05.291099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:06:05.307668 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:05.308737 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:05.309905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:06:05.309905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:06:05.309905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:06:05.309905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:05.309905 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 650/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:05.314518 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:05.407790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ca3b6-b1d0-43d5-bf79-abc5abf6af3d None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:05.438659 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:05.439297 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:05.439570 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:05.439838 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] PATH_INFO: `/v3/domains` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:05.440547 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:05.440846 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:05.442250 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:05.453108 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 629/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:06:05.459075 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:05.551814 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0ced2-a17e-44a7-9d14-225e7797ec49 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:05.582421 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:05.583177 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:05.583453 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:05.583721 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:05.590680 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:05.592095 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:06:05.609927 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:05.611300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:05.612604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 651/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:06:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:05.616798 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:05.710846 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ffe73-033a-4435-86d3-c89f2c510e51 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:05.741699 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:05.742344 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:05.742621 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:05.742889 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] PATH_INFO: `/v3/users` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:05.743709 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:05.744011 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:05.745454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:05.762184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 630/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:06:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:06:05.767740 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:05.862680 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d620fc6-924c-48dc-a0bb-49dd965258ef None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:05.894859 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0348f-8dcf-45f4-b173-0c95fb19e265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:05.895480 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0348f-8dcf-45f4-b173-0c95fb19e265 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:05.895762 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0348f-8dcf-45f4-b173-0c95fb19e265 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:05.896036 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0348f-8dcf-45f4-b173-0c95fb19e265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:05.958815 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0348f-8dcf-45f4-b173-0c95fb19e265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:05.960336 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0348f-8dcf-45f4-b173-0c95fb19e265 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:06.051673 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 652/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:06:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:06:06.057208 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:06.149325 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2bf7a-e971-499f-af2c-4bd82ec18664 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:06.180619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:06.181416 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:06.181694 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:06.181963 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:06.198239 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:06.199678 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:06:06.216476 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:06.217499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:06.218513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:06:06.218513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:06:06.218513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:06:06.218513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:06:06.218513 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 631/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:06.224108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0348f-8dcf-45f4-b173-0c95fb19e265 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:06.319460 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0348f-8dcf-45f4-b173-0c95fb19e265 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:06.350223 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:06.350856 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:06.351131 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:06.351399 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] PATH_INFO: `/v3/domains` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:06.352102 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:06.352403 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:06.353774 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:06.365811 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 653/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:06.371428 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:06.466574 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff656ba-9b73-4daf-800c-924051708d2e None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:06.499826 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:06.500589 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:06.500869 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:06.501172 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:06.507517 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:06.508951 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:06:06.522232 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application with Lock( May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:06.523131 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: ERROR keystone.server.flask.application May 09 15:06:06.524080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 632/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:06.529156 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:06.623512 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240189-f026-43fa-a4aa-9fad6a5dc781 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:06.655839 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5702c1e9-abf3-405a-ada4-3a9f603edc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:06.656472 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702c1e9-abf3-405a-ada4-3a9f603edc5b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:06.656759 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702c1e9-abf3-405a-ada4-3a9f603edc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:06.657058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5702c1e9-abf3-405a-ada4-3a9f603edc5b None admin] PATH_INFO: `/v3/projects` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:06.657841 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702c1e9-abf3-405a-ada4-3a9f603edc5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:06.658150 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702c1e9-abf3-405a-ada4-3a9f603edc5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:06.659567 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5702c1e9-abf3-405a-ada4-3a9f603edc5b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:06.671088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 654/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:06.676896 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:06.770554 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c82f9b-3699-4ca4-bcd7-4c86a527499b None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:06.801247 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5a288-9e65-4d89-8227-c2e405e29575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:06.801867 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a288-9e65-4d89-8227-c2e405e29575 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:06.802145 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a288-9e65-4d89-8227-c2e405e29575 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:06.802418 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a288-9e65-4d89-8227-c2e405e29575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:06.803230 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5a288-9e65-4d89-8227-c2e405e29575 None admin] RBAC: Adding query filter params (role.id=c73ae1fee2a2491482532db2e578cc32, scope.project.id=46e5f537a906431ba5baa049a20664df, user.id=f32b7537dc534f9da356c4230e27ef8e) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:06.803538 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5a288-9e65-4d89-8227-c2e405e29575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:06.804931 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5a288-9e65-4d89-8227-c2e405e29575 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:06.810496 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 633/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:06:06 2022] GET /identity/v3/role_assignments?user.id=f32b7537dc534f9da356c4230e27ef8e&scope.project.id=46e5f537a906431ba5baa049a20664df&role.id=c73ae1fee2a2491482532db2e578cc32 => generated 608 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:09.087996 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce5afa-070b-426e-b5c4-ea287c05b6c5 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:09.088550 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce5afa-070b-426e-b5c4-ea287c05b6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:09.088550 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce5afa-070b-426e-b5c4-ea287c05b6c5 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:09.089923 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 655/1288] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:06:09.095125 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392622d0-5053-4b39-a334-721f8a18870f None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:09.095522 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392622d0-5053-4b39-a334-721f8a18870f None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:09.095685 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392622d0-5053-4b39-a334-721f8a18870f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:09.112424 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-392622d0-5053-4b39-a334-721f8a18870f None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:06:09.113628 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-392622d0-5053-4b39-a334-721f8a18870f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:06:09.176276 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 634/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:06:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:06:09.198139 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce5afa-070b-426e-b5c4-ea287c05b6c5 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:09.325540 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce5afa-070b-426e-b5c4-ea287c05b6c5 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:09.356374 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:09.357116 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:09.357396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:09.357670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:09.358307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:09.359656 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:09.371306 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:09.377110 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:09.378997 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 656/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:06:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:09.380080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392622d0-5053-4b39-a334-721f8a18870f None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:09.471012 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392622d0-5053-4b39-a334-721f8a18870f None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:09.502739 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1bdd3-f416-4ba6-9c2e-35d5c9f9dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:09.503382 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1bdd3-f416-4ba6-9c2e-35d5c9f9dd47 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:09.503657 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1bdd3-f416-4ba6-9c2e-35d5c9f9dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:09.503925 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1bdd3-f416-4ba6-9c2e-35d5c9f9dd47 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:09.504617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1bdd3-f416-4ba6-9c2e-35d5c9f9dd47 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:09.504918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1bdd3-f416-4ba6-9c2e-35d5c9f9dd47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:09.506289 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1bdd3-f416-4ba6-9c2e-35d5c9f9dd47 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:09.510793 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 635/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:06:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:09.516390 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:09.623689 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2873e7-cf9b-4ba3-9115-870ed1d41e42 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:09.655162 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec6a67-c4ac-4636-9b03-c4b66176105b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:09.655808 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec6a67-c4ac-4636-9b03-c4b66176105b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:09.656088 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec6a67-c4ac-4636-9b03-c4b66176105b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:09.656363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec6a67-c4ac-4636-9b03-c4b66176105b None admin] PATH_INFO: `/v3/services` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:09.657082 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec6a67-c4ac-4636-9b03-c4b66176105b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:09.657396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec6a67-c4ac-4636-9b03-c4b66176105b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:09.658772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec6a67-c4ac-4636-9b03-c4b66176105b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:09.663481 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 657/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:06:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:11.845689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b31f11-5761-4889-b229-c6ecb9b6b03a None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:11.846227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b31f11-5761-4889-b229-c6ecb9b6b03a None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:11.846227 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b31f11-5761-4889-b229-c6ecb9b6b03a None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:11.848146 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 636/1293] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:06:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:06:11.853380 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817b056-53fb-4ad5-9e0c-e6a7a74304da None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:11.853797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817b056-53fb-4ad5-9e0c-e6a7a74304da None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:11.853931 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817b056-53fb-4ad5-9e0c-e6a7a74304da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:11.871432 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-a817b056-53fb-4ad5-9e0c-e6a7a74304da None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:06:11.872602 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-a817b056-53fb-4ad5-9e0c-e6a7a74304da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:06:11.934564 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 658/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:06:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:06:11.956326 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b31f11-5761-4889-b229-c6ecb9b6b03a None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:12.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b31f11-5761-4889-b229-c6ecb9b6b03a None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:12.081822 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7008f26-cb26-4ab6-a8c0-0dcb40c3386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:12.082469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7008f26-cb26-4ab6-a8c0-0dcb40c3386c None admin] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:12.082743 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7008f26-cb26-4ab6-a8c0-0dcb40c3386c None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:12.083012 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7008f26-cb26-4ab6-a8c0-0dcb40c3386c None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:12.083726 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7008f26-cb26-4ab6-a8c0-0dcb40c3386c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:12.085080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7008f26-cb26-4ab6-a8c0-0dcb40c3386c None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:12.097805 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 637/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:06:11 2022] POST /identity/v3/services => generated 246 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:06:14.300517 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0bc01-2ff1-4693-afc7-d853ae0e650c None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:14.301175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0bc01-2ff1-4693-afc7-d853ae0e650c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:14.301175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0bc01-2ff1-4693-afc7-d853ae0e650c None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:14.302461 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 659/1296] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:06:14.308372 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec405bb3-c414-4d04-b3ce-b5ce18a91af1 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:14.308905 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec405bb3-c414-4d04-b3ce-b5ce18a91af1 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:14.309009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec405bb3-c414-4d04-b3ce-b5ce18a91af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:14.326587 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-ec405bb3-c414-4d04-b3ce-b5ce18a91af1 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:06:14.327777 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-ec405bb3-c414-4d04-b3ce-b5ce18a91af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:06:14.408918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 638/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:06:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:06:14.431722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0bc01-2ff1-4693-afc7-d853ae0e650c None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:14.530012 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0bc01-2ff1-4693-afc7-d853ae0e650c None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:14.562018 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:14.562782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:14.563058 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:14.563324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:14.563958 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:14.565365 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:14.577140 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:14.578309 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:14.579203 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 660/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:06:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:14.584333 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec405bb3-c414-4d04-b3ce-b5ce18a91af1 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:14.680790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec405bb3-c414-4d04-b3ce-b5ce18a91af1 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:14.711200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:14.711843 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:14.712117 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:14.712385 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:14.713095 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:14.713398 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:14.714747 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:14.719580 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 639/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:06:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:14.725981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:14.819653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219eba9-18af-41d2-97c3-a254f91abbb1 None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:14.850806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ebf27-ac1a-4645-988c-a81318f2e3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:14.851430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebf27-ac1a-4645-988c-a81318f2e3bc None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:14.851706 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebf27-ac1a-4645-988c-a81318f2e3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:14.851976 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebf27-ac1a-4645-988c-a81318f2e3bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:14.852762 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebf27-ac1a-4645-988c-a81318f2e3bc None admin] RBAC: Adding query filter params (interface=public, service_id=e4568409e25b47e788882575d8eae115, region_id=RegionOne) {{(pid=80062) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:14.853094 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebf27-ac1a-4645-988c-a81318f2e3bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:14.854439 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebf27-ac1a-4645-988c-a81318f2e3bc None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:14.859267 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 661/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:06:14 2022] GET /identity/v3/endpoints?service_id=e4568409e25b47e788882575d8eae115&interface=public®ion_id=RegionOne => generated 194 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:14.865262 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:14.958835 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b39299-7650-4f46-ad12-9720f16d4749 None admin] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:14.990992 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e9b74-e7a6-427e-a7ab-0979e0796621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:14.991635 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e9b74-e7a6-427e-a7ab-0979e0796621 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:14.991912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e9b74-e7a6-427e-a7ab-0979e0796621 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:14.992182 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e9b74-e7a6-427e-a7ab-0979e0796621 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:14.992852 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e9b74-e7a6-427e-a7ab-0979e0796621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:14.994231 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e9b74-e7a6-427e-a7ab-0979e0796621 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:15.000977 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 640/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:06:14 2022] GET /identity/v3/services => generated 2191 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:06:17.199338 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e555c-bf55-49c1-b9db-baf885f0de78 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:17.199847 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e555c-bf55-49c1-b9db-baf885f0de78 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:17.199936 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e555c-bf55-49c1-b9db-baf885f0de78 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:17.201260 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 662/1302] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:06:17.206428 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67483f8-e675-4a61-94b9-411ac974ba34 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:17.206787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67483f8-e675-4a61-94b9-411ac974ba34 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:17.206974 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67483f8-e675-4a61-94b9-411ac974ba34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:17.223485 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-c67483f8-e675-4a61-94b9-411ac974ba34 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:06:17.224642 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-c67483f8-e675-4a61-94b9-411ac974ba34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:06:17.286552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 641/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:06:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:06:17.309005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e555c-bf55-49c1-b9db-baf885f0de78 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:17.403510 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e555c-bf55-49c1-b9db-baf885f0de78 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:17.434401 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:17.435167 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:17.435444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:17.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:17.436348 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:17.437722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:06:17.449053 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:06:17.450235 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application with Lock( May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self._enter() May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application return self.creator() May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application created_value = creator( May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: ERROR keystone.server.flask.application May 09 15:06:17.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 663/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:06:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:06:17.456554 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67483f8-e675-4a61-94b9-411ac974ba34 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:17.549200 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67483f8-e675-4a61-94b9-411ac974ba34 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:17.580499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d84020-74a5-4536-ab8a-e0aa32fb4526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:17.581709 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d84020-74a5-4536-ab8a-e0aa32fb4526 None admin] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:17.581993 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d84020-74a5-4536-ab8a-e0aa32fb4526 None admin] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:17.582268 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d84020-74a5-4536-ab8a-e0aa32fb4526 None admin] PATH_INFO: `/v3/services` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:17.582969 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d84020-74a5-4536-ab8a-e0aa32fb4526 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80060) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:06:17.583275 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d84020-74a5-4536-ab8a-e0aa32fb4526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:17.584623 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d84020-74a5-4536-ab8a-e0aa32fb4526 None admin] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:17.589545 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 642/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:06:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:06:17.595184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:06:17.693722 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afd197-fcec-4176-a9bd-5a36038b5f7b None admin] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:06:17.725300 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90861069-2f40-48d3-8044-39a5b117c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c980ae99c7444e0390a8a5607832c346', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:06:17.725934 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90861069-2f40-48d3-8044-39a5b117c4ac None admin] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:06:17.726211 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90861069-2f40-48d3-8044-39a5b117c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:06:17.726483 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90861069-2f40-48d3-8044-39a5b117c4ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:06:17.727214 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90861069-2f40-48d3-8044-39a5b117c4ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:06:17.728571 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90861069-2f40-48d3-8044-39a5b117c4ac None admin] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:06:17.749452 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 664/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:06:17 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:07:21.107546 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1bce5-8680-4ecc-99f5-66e753948912 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:21.108466 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1bce5-8680-4ecc-99f5-66e753948912 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:21.108912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1bce5-8680-4ecc-99f5-66e753948912 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:21.116396 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab322b3-cb50-4232-aa02-c7aab8c3fbe3 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:21.117253 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab322b3-cb50-4232-aa02-c7aab8c3fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:21.117688 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab322b3-cb50-4232-aa02-c7aab8c3fbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:21.121254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 643/1307] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:21 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:21.194880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-eab322b3-cb50-4232-aa02-c7aab8c3fbe3 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:21.196327 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-eab322b3-cb50-4232-aa02-c7aab8c3fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:21.355320 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 665/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:07:21.407081 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ad1af-9f8f-4e9d-a5e4-c822650fe98c None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:21.408339 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ad1af-9f8f-4e9d-a5e4-c822650fe98c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:21.408991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ad1af-9f8f-4e9d-a5e4-c822650fe98c None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:21.412030 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 644/1309] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 9 15:07:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:21.417962 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c14b12-ff5a-4afe-8dff-05463a2b274d None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:21.418245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c14b12-ff5a-4afe-8dff-05463a2b274d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:21.418961 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c14b12-ff5a-4afe-8dff-05463a2b274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:21.522194 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-61c14b12-ff5a-4afe-8dff-05463a2b274d None None] MFA Rules not processed for user `adf51a2d99154eccbc74f0c63fc18bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:21.523429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-61c14b12-ff5a-4afe-8dff-05463a2b274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:21.604046 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 666/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 9 15:07:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:21.612832 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ad1af-9f8f-4e9d-a5e4-c822650fe98c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:21.614779 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-5b2ad1af-9f8f-4e9d-a5e4-c822650fe98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:21.728955 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ad1af-9f8f-4e9d-a5e4-c822650fe98c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:21.803989 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8dd5a-e0d1-4241-a6e0-50bdde991e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:21.804633 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8dd5a-e0d1-4241-a6e0-50bdde991e3a service neutron] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:21.804914 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8dd5a-e0d1-4241-a6e0-50bdde991e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:21.805222 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8dd5a-e0d1-4241-a6e0-50bdde991e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:21.806878 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-44d8dd5a-e0d1-4241-a6e0-50bdde991e3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:21.916991 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8dd5a-e0d1-4241-a6e0-50bdde991e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:21.918458 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8dd5a-e0d1-4241-a6e0-50bdde991e3a service neutron] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:21.936534 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 645/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:24.464827 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55710-2d9c-40fd-87b3-a4e26484aa4f None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:24.465429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55710-2d9c-40fd-87b3-a4e26484aa4f None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:24.465429 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55710-2d9c-40fd-87b3-a4e26484aa4f None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:24.466782 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 667/1312] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:24.472039 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bcff1-3108-4623-830d-0a59e2879917 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:24.472477 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bcff1-3108-4623-830d-0a59e2879917 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:24.472686 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bcff1-3108-4623-830d-0a59e2879917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:24.552898 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-ae6bcff1-3108-4623-830d-0a59e2879917 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:24.554063 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-ae6bcff1-3108-4623-830d-0a59e2879917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:24.646882 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 646/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:24.684571 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec55710-2d9c-40fd-87b3-a4e26484aa4f None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:24.686210 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-3ec55710-2d9c-40fd-87b3-a4e26484aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:24.790787 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec55710-2d9c-40fd-87b3-a4e26484aa4f None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:24.863824 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039150b-416c-4af2-8ce4-e7385b57a5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:24.864479 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039150b-416c-4af2-8ce4-e7385b57a5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:24.864767 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039150b-416c-4af2-8ce4-e7385b57a5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:24.865059 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039150b-416c-4af2-8ce4-e7385b57a5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:24.866714 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-b039150b-416c-4af2-8ce4-e7385b57a5d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:24.971185 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039150b-416c-4af2-8ce4-e7385b57a5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:24.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039150b-416c-4af2-8ce4-e7385b57a5d2 service neutron] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:24.990471 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 668/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:27.538052 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280572f2-aa52-47d8-8b74-47e2fff0963d None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:27.538610 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280572f2-aa52-47d8-8b74-47e2fff0963d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:27.538610 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280572f2-aa52-47d8-8b74-47e2fff0963d None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:27.539966 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 647/1315] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:27.545223 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab98b5-d8ca-4a73-b06e-b945607541f7 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:27.545627 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab98b5-d8ca-4a73-b06e-b945607541f7 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:27.545772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab98b5-d8ca-4a73-b06e-b945607541f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:27.607085 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-a7ab98b5-d8ca-4a73-b06e-b945607541f7 None None] MFA Rules not processed for user `6a6136a2abe6402cab688d9bb754142c`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:27.608273 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-a7ab98b5-d8ca-4a73-b06e-b945607541f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:27.706703 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 669/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:27.746278 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280572f2-aa52-47d8-8b74-47e2fff0963d None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:27.748184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-280572f2-aa52-47d8-8b74-47e2fff0963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:27.854773 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280572f2-aa52-47d8-8b74-47e2fff0963d None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:27.929698 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1d4e0-29a3-47d8-9926-705a64ea4e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:27.930349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d4e0-29a3-47d8-9926-705a64ea4e2f service neutron] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:27.930627 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d4e0-29a3-47d8-9926-705a64ea4e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:27.930899 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d4e0-29a3-47d8-9926-705a64ea4e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:27.932544 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-25a1d4e0-29a3-47d8-9926-705a64ea4e2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:28.036600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1d4e0-29a3-47d8-9926-705a64ea4e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:28.038064 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1d4e0-29a3-47d8-9926-705a64ea4e2f service neutron] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:28.055837 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 648/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:30.836650 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9416b23-c91e-4db1-ad30-08826ec6eaca None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:30.837243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9416b23-c91e-4db1-ad30-08826ec6eaca None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:30.837243 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9416b23-c91e-4db1-ad30-08826ec6eaca None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:30.838606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 670/1318] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:30.843876 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114e3c-1a17-4ca9-9563-951b7e85d718 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:30.844336 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114e3c-1a17-4ca9-9563-951b7e85d718 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:30.845061 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70114e3c-1a17-4ca9-9563-951b7e85d718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:30.907321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-70114e3c-1a17-4ca9-9563-951b7e85d718 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:30.908486 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-70114e3c-1a17-4ca9-9563-951b7e85d718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:30.999173 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 649/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:31.037982 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9416b23-c91e-4db1-ad30-08826ec6eaca None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:31.039515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-d9416b23-c91e-4db1-ad30-08826ec6eaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:31.139670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9416b23-c91e-4db1-ad30-08826ec6eaca None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:31.209981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cb258-7b04-473c-a485-71d4953df100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:31.210615 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb258-7b04-473c-a485-71d4953df100 service neutron] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:31.210897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb258-7b04-473c-a485-71d4953df100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:31.211173 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb258-7b04-473c-a485-71d4953df100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:31.212807 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-d57cb258-7b04-473c-a485-71d4953df100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:31.316642 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cb258-7b04-473c-a485-71d4953df100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:31.318098 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cb258-7b04-473c-a485-71d4953df100 service neutron] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:31.335538 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 671/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:34.200844 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f516-d350-4f09-b9c8-7f3da74909cd None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:34.201435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f516-d350-4f09-b9c8-7f3da74909cd None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:34.201435 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f516-d350-4f09-b9c8-7f3da74909cd None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:34.202790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 650/1321] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:07:34.208066 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a931fd-772f-429b-8b59-39e047dca3f5 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:34.208484 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a931fd-772f-429b-8b59-39e047dca3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:34.208628 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a931fd-772f-429b-8b59-39e047dca3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:34.272947 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-f3a931fd-772f-429b-8b59-39e047dca3f5 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:34.274175 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-f3a931fd-772f-429b-8b59-39e047dca3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:34.367515 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 672/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:34.402454 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1f516-d350-4f09-b9c8-7f3da74909cd None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:34.403965 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-b6a1f516-d350-4f09-b9c8-7f3da74909cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:34.504687 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1f516-d350-4f09-b9c8-7f3da74909cd None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:34.575400 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fb93e-fe0a-4433-aa14-855bdefbb682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:34.576042 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb93e-fe0a-4433-aa14-855bdefbb682 service neutron] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:34.576321 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb93e-fe0a-4433-aa14-855bdefbb682 service neutron] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:34.576598 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fb93e-fe0a-4433-aa14-855bdefbb682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:34.578265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-e48fb93e-fe0a-4433-aa14-855bdefbb682 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:34.684293 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fb93e-fe0a-4433-aa14-855bdefbb682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:34.685816 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fb93e-fe0a-4433-aa14-855bdefbb682 service neutron] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:34.703037 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 651/1323] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:36.993691 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb89d6-6281-4596-8383-25ba17844b48 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:36.994230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb89d6-6281-4596-8383-25ba17844b48 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:36.994230 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb89d6-6281-4596-8383-25ba17844b48 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:36.995609 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 673/1324] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:37.001207 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28cfa-5a6b-461b-9a82-7e809f133c96 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:37.001619 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28cfa-5a6b-461b-9a82-7e809f133c96 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:37.001741 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28cfa-5a6b-461b-9a82-7e809f133c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:37.067885 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-a2b28cfa-5a6b-461b-9a82-7e809f133c96 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:37.069079 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-a2b28cfa-5a6b-461b-9a82-7e809f133c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:37.163778 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 652/1325] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:37.212507 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9278-fcd9-4ebc-bb77-e3573416d8fb None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:37.212795 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9278-fcd9-4ebc-bb77-e3573416d8fb None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:37.213108 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9278-fcd9-4ebc-bb77-e3573416d8fb None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:37.214507 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 674/1326] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 9 15:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:37.220930 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac722e-9354-44e3-b9c6-fecdf3802241 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:37.221266 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac722e-9354-44e3-b9c6-fecdf3802241 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:37.221544 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac722e-9354-44e3-b9c6-fecdf3802241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:37.296584 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-ddac722e-9354-44e3-b9c6-fecdf3802241 None None] MFA Rules not processed for user `adf51a2d99154eccbc74f0c63fc18bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:37.298248 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-ddac722e-9354-44e3-b9c6-fecdf3802241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:37.369944 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 653/1327] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 9 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:37.379324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9278-fcd9-4ebc-bb77-e3573416d8fb None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:37.380834 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-a24f9278-fcd9-4ebc-bb77-e3573416d8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:37.479048 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9278-fcd9-4ebc-bb77-e3573416d8fb None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:37.550183 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91591a0-54b4-4127-bdd4-15e3a472b0da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:37.550786 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91591a0-54b4-4127-bdd4-15e3a472b0da service neutron] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:37.551075 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91591a0-54b4-4127-bdd4-15e3a472b0da service neutron] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:37.551349 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91591a0-54b4-4127-bdd4-15e3a472b0da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:37.552979 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-d91591a0-54b4-4127-bdd4-15e3a472b0da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:37.658690 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91591a0-54b4-4127-bdd4-15e3a472b0da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:37.660138 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91591a0-54b4-4127-bdd4-15e3a472b0da service neutron] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:37.677700 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 675/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:40.180456 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccde885-2912-4c7f-843b-51bf8138e15c None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:40.180997 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccde885-2912-4c7f-843b-51bf8138e15c None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:40.180997 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccde885-2912-4c7f-843b-51bf8138e15c None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:40.182411 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 654/1329] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:40.187797 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f1c3a-985f-4053-aa7f-318b81b2f186 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:40.188225 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f1c3a-985f-4053-aa7f-318b81b2f186 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:40.188345 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f1c3a-985f-4053-aa7f-318b81b2f186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:40.257323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-e64f1c3a-985f-4053-aa7f-318b81b2f186 None None] MFA Rules not processed for user `6a6136a2abe6402cab688d9bb754142c`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:40.258531 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-e64f1c3a-985f-4053-aa7f-318b81b2f186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:40.351698 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 676/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:40.389279 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccde885-2912-4c7f-843b-51bf8138e15c None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:40.390871 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-fccde885-2912-4c7f-843b-51bf8138e15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:40.491065 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccde885-2912-4c7f-843b-51bf8138e15c None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:40.567788 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262f483-b616-4341-a787-b13854c09c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:40.568434 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262f483-b616-4341-a787-b13854c09c3b service neutron] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:40.568714 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262f483-b616-4341-a787-b13854c09c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:40.568987 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262f483-b616-4341-a787-b13854c09c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:40.570655 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-a262f483-b616-4341-a787-b13854c09c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:40.674683 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262f483-b616-4341-a787-b13854c09c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:40.676123 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262f483-b616-4341-a787-b13854c09c3b service neutron] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:40.693552 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 655/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:43.336738 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864100-e07b-45e2-b227-9ebcacbd6939 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:43.337293 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864100-e07b-45e2-b227-9ebcacbd6939 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:43.337394 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a864100-e07b-45e2-b227-9ebcacbd6939 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:43.338784 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 677/1332] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:43.344861 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f63bd-a2fc-4f08-8cf8-0e8bec871083 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:43.345332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f63bd-a2fc-4f08-8cf8-0e8bec871083 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:43.345591 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f63bd-a2fc-4f08-8cf8-0e8bec871083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:43.415902 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-943f63bd-a2fc-4f08-8cf8-0e8bec871083 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:43.417368 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-943f63bd-a2fc-4f08-8cf8-0e8bec871083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:43.515172 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 656/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:43.552573 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a864100-e07b-45e2-b227-9ebcacbd6939 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:43.554191 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-7a864100-e07b-45e2-b227-9ebcacbd6939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:43.658998 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a864100-e07b-45e2-b227-9ebcacbd6939 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:43.729717 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4000ca-dd10-40ee-9b36-2466fe51a5ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:43.730362 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4000ca-dd10-40ee-9b36-2466fe51a5ce service neutron] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:43.730640 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4000ca-dd10-40ee-9b36-2466fe51a5ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:43.730910 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4000ca-dd10-40ee-9b36-2466fe51a5ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:43.732537 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-fc4000ca-dd10-40ee-9b36-2466fe51a5ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:43.836880 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4000ca-dd10-40ee-9b36-2466fe51a5ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:43.838351 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4000ca-dd10-40ee-9b36-2466fe51a5ce service neutron] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:43.856213 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 678/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:46.153197 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1db6ab-69c8-45ce-a8c9-748a3bc61fd9 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:46.153755 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1db6ab-69c8-45ce-a8c9-748a3bc61fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:46.153755 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1db6ab-69c8-45ce-a8c9-748a3bc61fd9 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:46.155092 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 657/1335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:46.160269 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5de93-9038-4c15-bab7-b6b4bf872ea7 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:46.160708 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5de93-9038-4c15-bab7-b6b4bf872ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:46.160817 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5de93-9038-4c15-bab7-b6b4bf872ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:46.222771 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-19a5de93-9038-4c15-bab7-b6b4bf872ea7 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:46.223984 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-19a5de93-9038-4c15-bab7-b6b4bf872ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:46.317469 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 679/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:46.356332 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1db6ab-69c8-45ce-a8c9-748a3bc61fd9 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:46.357884 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-5f1db6ab-69c8-45ce-a8c9-748a3bc61fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:46.455870 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1db6ab-69c8-45ce-a8c9-748a3bc61fd9 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:46.524158 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b6217-00d8-4276-8985-bc96a963d3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:46.524787 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6217-00d8-4276-8985-bc96a963d3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:46.525080 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6217-00d8-4276-8985-bc96a963d3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:46.525349 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b6217-00d8-4276-8985-bc96a963d3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:46.526950 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-2f2b6217-00d8-4276-8985-bc96a963d3e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:46.630571 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b6217-00d8-4276-8985-bc96a963d3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:46.631984 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b6217-00d8-4276-8985-bc96a963d3e2 service neutron] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:46.649040 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 658/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:49.353525 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261512c6-2298-46cf-9580-42032ead9166 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:49.354041 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261512c6-2298-46cf-9580-42032ead9166 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:49.354041 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261512c6-2298-46cf-9580-42032ead9166 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:49.355369 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 680/1338] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:49.360478 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326dd6-0372-42b2-bf0b-4ef3308710f7 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:49.360881 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326dd6-0372-42b2-bf0b-4ef3308710f7 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:49.361025 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326dd6-0372-42b2-bf0b-4ef3308710f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:49.422767 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-eb326dd6-0372-42b2-bf0b-4ef3308710f7 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:49.423943 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-eb326dd6-0372-42b2-bf0b-4ef3308710f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:49.514725 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 659/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:49.549699 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261512c6-2298-46cf-9580-42032ead9166 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:49.551221 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-261512c6-2298-46cf-9580-42032ead9166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:49.648953 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261512c6-2298-46cf-9580-42032ead9166 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:49.718405 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe9817-8c28-4327-8a6b-ce0076f9b37d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:49.719037 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9817-8c28-4327-8a6b-ce0076f9b37d service neutron] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:49.719314 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9817-8c28-4327-8a6b-ce0076f9b37d service neutron] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:49.719581 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe9817-8c28-4327-8a6b-ce0076f9b37d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:49.721248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-c2fe9817-8c28-4327-8a6b-ce0076f9b37d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:49.826336 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe9817-8c28-4327-8a6b-ce0076f9b37d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:49.827799 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe9817-8c28-4327-8a6b-ce0076f9b37d service neutron] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:49.845054 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 681/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:54.083289 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d51251-6c34-4367-85fb-8904f91d84df None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:54.083861 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d51251-6c34-4367-85fb-8904f91d84df None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:54.083861 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d51251-6c34-4367-85fb-8904f91d84df None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:54.085122 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 660/1341] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:54.090587 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd7d2-c441-4267-babe-ca3ac3f0d383 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:54.090981 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd7d2-c441-4267-babe-ca3ac3f0d383 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:54.091270 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcd7d2-c441-4267-babe-ca3ac3f0d383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:54.160550 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-eefcd7d2-c441-4267-babe-ca3ac3f0d383 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:54.161753 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-eefcd7d2-c441-4267-babe-ca3ac3f0d383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:54.262747 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 682/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:54.301468 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d51251-6c34-4367-85fb-8904f91d84df None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:54.303612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-62d51251-6c34-4367-85fb-8904f91d84df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:54.399702 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d51251-6c34-4367-85fb-8904f91d84df None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:54.404680 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e245da-6d45-42c6-af1f-fce0c717dca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:54.405480 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e245da-6d45-42c6-af1f-fce0c717dca7 service neutron] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:54.405794 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e245da-6d45-42c6-af1f-fce0c717dca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:54.406245 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e245da-6d45-42c6-af1f-fce0c717dca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:54.415688 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e245da-6d45-42c6-af1f-fce0c717dca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:54.417162 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e245da-6d45-42c6-af1f-fce0c717dca7 service neutron] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:54.426636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 661/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:07:57.266344 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e3fe5-1041-4b2e-b112-bcf958622aea None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:57.266903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e3fe5-1041-4b2e-b112-bcf958622aea None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:57.266903 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e3fe5-1041-4b2e-b112-bcf958622aea None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:57.268184 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 683/1344] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:07:57.273457 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ab7a-3eb6-41b5-bf6c-d501c71ce4a9 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:57.273868 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ab7a-3eb6-41b5-bf6c-d501c71ce4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:57.274009 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ab7a-3eb6-41b5-bf6c-d501c71ce4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:57.300918 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-12c7ab7a-3eb6-41b5-bf6c-d501c71ce4a9 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:07:57.302100 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-12c7ab7a-3eb6-41b5-bf6c-d501c71ce4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:07:57.381569 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 662/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:07:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:07:57.417966 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e3fe5-1041-4b2e-b112-bcf958622aea None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:07:57.429187 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e3fe5-1041-4b2e-b112-bcf958622aea None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:07:57.432665 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde5818-ec0e-43b2-8c5f-456e594860b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:07:57.433327 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde5818-ec0e-43b2-8c5f-456e594860b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:07:57.433609 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde5818-ec0e-43b2-8c5f-456e594860b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:07:57.433883 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde5818-ec0e-43b2-8c5f-456e594860b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:07:57.442653 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde5818-ec0e-43b2-8c5f-456e594860b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:07:57.444075 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde5818-ec0e-43b2-8c5f-456e594860b8 service neutron] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:07:57.453430 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 684/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:07:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:08:01.962555 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf210-6934-489c-b927-89f9787b42a4 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:01.963095 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf210-6934-489c-b927-89f9787b42a4 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:01.963095 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf210-6934-489c-b927-89f9787b42a4 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:01.964371 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 663/1347] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:01.969731 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb1803-160a-4bca-9ac4-c9a8ef5cbdf2 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:01.970117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb1803-160a-4bca-9ac4-c9a8ef5cbdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:01.970294 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb1803-160a-4bca-9ac4-c9a8ef5cbdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:01.989670 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-4cfb1803-160a-4bca-9ac4-c9a8ef5cbdf2 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:01.990851 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-4cfb1803-160a-4bca-9ac4-c9a8ef5cbdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:01.998933 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 685/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:02.037052 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbf210-6934-489c-b927-89f9787b42a4 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:08:02.047238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbf210-6934-489c-b927-89f9787b42a4 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:08:02.050689 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1ccbe-3849-4327-8080-156984a4fe61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adf51a2d99154eccbc74f0c63fc18bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:08:02.051313 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1ccbe-3849-4327-8080-156984a4fe61 service neutron] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:02.051588 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1ccbe-3849-4327-8080-156984a4fe61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:02.051909 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1ccbe-3849-4327-8080-156984a4fe61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:02.060430 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1ccbe-3849-4327-8080-156984a4fe61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:08:02.062375 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1ccbe-3849-4327-8080-156984a4fe61 service neutron] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:08:02.071445 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 664/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:08:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:08:07.461323 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4a9d-49fe-4198-aab6-1a04dc06152c None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:07.461828 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4a9d-49fe-4198-aab6-1a04dc06152c None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:07.461828 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec4a9d-49fe-4198-aab6-1a04dc06152c None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:07.463161 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 686/1350] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:07.468756 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f2ad-cf3a-4fe5-8155-83adaa3f700e None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:07.469617 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f2ad-cf3a-4fe5-8155-83adaa3f700e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:07.470066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f2ad-cf3a-4fe5-8155-83adaa3f700e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:07.540571 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-7460f2ad-cf3a-4fe5-8155-83adaa3f700e None None] MFA Rules not processed for user `f32b7537dc534f9da356c4230e27ef8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:07.542163 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-7460f2ad-cf3a-4fe5-8155-83adaa3f700e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:07.627906 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 665/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:08:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:11.955972 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f22a6-e3df-4c73-b16e-318410cc160d None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:11.956559 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f22a6-e3df-4c73-b16e-318410cc160d None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:11.956559 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f22a6-e3df-4c73-b16e-318410cc160d None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:11.958248 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 687/1352] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:11.963510 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2d743-3918-4176-b133-b94ef4e342af None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:11.963982 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2d743-3918-4176-b133-b94ef4e342af None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:11.964066 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2d743-3918-4176-b133-b94ef4e342af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:11.982694 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-f6b2d743-3918-4176-b133-b94ef4e342af None None] MFA Rules not processed for user `f32b7537dc534f9da356c4230e27ef8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:11.984607 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-f6b2d743-3918-4176-b133-b94ef4e342af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:11.993201 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 666/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:08:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:13.623341 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162d2d-67aa-4d84-a540-1d7be1246630 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:13.623874 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162d2d-67aa-4d84-a540-1d7be1246630 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:13.623985 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162d2d-67aa-4d84-a540-1d7be1246630 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:13.625297 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 688/1354] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:13.630104 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f65c9-6d03-4742-b488-1e9b93406f7d None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:13.630499 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f65c9-6d03-4742-b488-1e9b93406f7d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:13.630646 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f65c9-6d03-4742-b488-1e9b93406f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:13.648585 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-1a0f65c9-6d03-4742-b488-1e9b93406f7d None None] MFA Rules not processed for user `f32b7537dc534f9da356c4230e27ef8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:13.649775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-1a0f65c9-6d03-4742-b488-1e9b93406f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:13.658759 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 667/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:08:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:08:17.515781 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78653393-2355-476c-b94f-67ad1bd58cf9 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:17.516368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78653393-2355-476c-b94f-67ad1bd58cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:17.516368 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78653393-2355-476c-b94f-67ad1bd58cf9 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:17.518896 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 689/1356] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:08:17.523103 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519080-337e-4cd1-998a-5600cc05628d None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:17.523912 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519080-337e-4cd1-998a-5600cc05628d None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:17.524391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519080-337e-4cd1-998a-5600cc05628d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:17.543612 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-e2519080-337e-4cd1-998a-5600cc05628d None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:17.544940 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-e2519080-337e-4cd1-998a-5600cc05628d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:17.565733 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 668/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:17.602958 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dece70a-bf28-4a00-860e-0c549147de72 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:17.603307 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dece70a-bf28-4a00-860e-0c549147de72 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:17.603651 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dece70a-bf28-4a00-860e-0c549147de72 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:17.605136 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 690/1358] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 9 15:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:17.611184 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca5a0-c8e7-47dc-acc4-e9f4b04f5bb9 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:17.611469 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca5a0-c8e7-47dc-acc4-e9f4b04f5bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:17.611737 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca5a0-c8e7-47dc-acc4-e9f4b04f5bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:17.659734 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-7ecca5a0-c8e7-47dc-acc4-e9f4b04f5bb9 None None] MFA Rules not processed for user `512c115af57c463eb1fbe636667306dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:17.660887 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-7ecca5a0-c8e7-47dc-acc4-e9f4b04f5bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:17.740775 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 669/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 9 15:08:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:08:17.748730 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dece70a-bf28-4a00-860e-0c549147de72 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:08:17.762973 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dece70a-bf28-4a00-860e-0c549147de72 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:08:17.768576 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce646dd-b1f0-4d1f-86dc-55b0fa5bf354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512c115af57c463eb1fbe636667306dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:08:17.769291 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce646dd-b1f0-4d1f-86dc-55b0fa5bf354 service nova] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:17.769706 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce646dd-b1f0-4d1f-86dc-55b0fa5bf354 service nova] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:17.770165 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce646dd-b1f0-4d1f-86dc-55b0fa5bf354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:17.778245 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce646dd-b1f0-4d1f-86dc-55b0fa5bf354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:08:17.779949 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce646dd-b1f0-4d1f-86dc-55b0fa5bf354 service nova] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:08:17.790324 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 691/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:08:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:08:20.304148 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c49f4-aa61-46a0-9c73-b1a1d35d08b1 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:20.304736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c49f4-aa61-46a0-9c73-b1a1d35d08b1 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:20.304736 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c49f4-aa61-46a0-9c73-b1a1d35d08b1 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:20.306122 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 670/1361] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:20.312117 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b424-c102-4db9-ac00-909ba797c102 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:20.312897 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b424-c102-4db9-ac00-909ba797c102 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:20.313380 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5b424-c102-4db9-ac00-909ba797c102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:20.331027 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-9ce5b424-c102-4db9-ac00-909ba797c102 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:20.332099 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-9ce5b424-c102-4db9-ac00-909ba797c102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:20.340290 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 692/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:20.385254 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c49f4-aa61-46a0-9c73-b1a1d35d08b1 None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:08:20.399391 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c49f4-aa61-46a0-9c73-b1a1d35d08b1 None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:08:20.405636 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbddd7-884b-4cc1-87b0-619de116b378 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512c115af57c463eb1fbe636667306dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:08:20.406265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbddd7-884b-4cc1-87b0-619de116b378 service nova] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:20.406543 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbddd7-884b-4cc1-87b0-619de116b378 service nova] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:20.406812 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbddd7-884b-4cc1-87b0-619de116b378 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:20.415596 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbddd7-884b-4cc1-87b0-619de116b378 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:08:20.417225 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbddd7-884b-4cc1-87b0-619de116b378 service nova] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:08:20.428259 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 671/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:08:22.964822 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e666-c074-440b-badd-a56a14735f7b None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:22.965428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e666-c074-440b-badd-a56a14735f7b None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:22.965428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967e666-c074-440b-badd-a56a14735f7b None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:22.966794 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 693/1364] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:22.972761 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb798-93bf-4434-b287-4caab776dfd9 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:22.973558 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb798-93bf-4434-b287-4caab776dfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:22.973937 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fb798-93bf-4434-b287-4caab776dfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:22.991908 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-9d8fb798-93bf-4434-b287-4caab776dfd9 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:22.993265 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-9d8fb798-93bf-4434-b287-4caab776dfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:23.001970 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 672/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:23.048195 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727ae6-9948-4136-842f-aeac4046d1d9 None None] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:23.048606 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727ae6-9948-4136-842f-aeac4046d1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:23.049005 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727ae6-9948-4136-842f-aeac4046d1d9 None None] PATH_INFO: `/` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:23.050772 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 694/1366] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 9 15:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:23.056098 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc95c0-d414-43e3-a877-a772aa0a0730 None None] REQUEST_METHOD: `POST` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:23.056666 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc95c0-d414-43e3-a877-a772aa0a0730 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:23.057082 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc95c0-d414-43e3-a877-a772aa0a0730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:23.075840 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.auth.core [None req-5cdc95c0-d414-43e3-a877-a772aa0a0730 None None] MFA Rules not processed for user `512c115af57c463eb1fbe636667306dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80060) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:23.077729 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.fernet_utils [None req-5cdc95c0-d414-43e3-a877-a772aa0a0730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80060) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:23.087370 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 673/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 9 15:08:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:23.095571 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca727ae6-9948-4136-842f-aeac4046d1d9 None None] Authenticating user token {{(pid=80062) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:08:23.110806 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca727ae6-9948-4136-842f-aeac4046d1d9 None None] Validating token access rules against request {{(pid=80062) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:08:23.119072 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892da3c-04da-4dd7-8a87-67ae7278107c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512c115af57c463eb1fbe636667306dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80062) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:08:23.120091 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892da3c-04da-4dd7-8a87-67ae7278107c service nova] REQUEST_METHOD: `GET` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:23.120598 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892da3c-04da-4dd7-8a87-67ae7278107c service nova] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:23.121119 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892da3c-04da-4dd7-8a87-67ae7278107c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:23.132428 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892da3c-04da-4dd7-8a87-67ae7278107c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:08:23.135011 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892da3c-04da-4dd7-8a87-67ae7278107c service nova] RBAC: Authorization granted {{(pid=80062) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:08:23.146164 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 695/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:08:23.913647 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6068d-8486-4b45-b475-40024f2c2930 None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:23.914335 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6068d-8486-4b45-b475-40024f2c2930 None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:23.914613 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6068d-8486-4b45-b475-40024f2c2930 None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:23.916165 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 674/1369] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:23.922170 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b6360-268e-4aa8-9009-f4f5a9bdb67e None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:23.922604 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b6360-268e-4aa8-9009-f4f5a9bdb67e None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:23.922983 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b6360-268e-4aa8-9009-f4f5a9bdb67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:23.943029 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-e78b6360-268e-4aa8-9009-f4f5a9bdb67e None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:23.944714 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-e78b6360-268e-4aa8-9009-f4f5a9bdb67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:23.954094 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 696/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:25.478480 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dd62a-ac6d-4bb4-acc9-12c79edaa59e None None] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:25.479021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dd62a-ac6d-4bb4-acc9-12c79edaa59e None None] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:25.479021 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4dd62a-ac6d-4bb4-acc9-12c79edaa59e None None] PATH_INFO: `/` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:25.480434 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 675/1371] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:08:25.486844 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d9bf-f76a-4743-8584-dec34696da46 None None] REQUEST_METHOD: `POST` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:25.487363 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d9bf-f76a-4743-8584-dec34696da46 None None] SCRIPT_NAME: `/identity` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:25.487444 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d9bf-f76a-4743-8584-dec34696da46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80062) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:25.506644 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.auth.core [None req-42e5d9bf-f76a-4743-8584-dec34696da46 None None] MFA Rules not processed for user `c980ae99c7444e0390a8a5607832c346`. Rule list: `[]` (Enabled: `True`). {{(pid=80062) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:08:25.507845 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: DEBUG keystone.common.fernet_utils [None req-42e5d9bf-f76a-4743-8584-dec34696da46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80062) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:08:25.516568 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: [pid: 80062|app: 0|req: 697/1372] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:08:25.564790 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dd62a-ac6d-4bb4-acc9-12c79edaa59e None None] Authenticating user token {{(pid=80060) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:08:25.580221 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4dd62a-ac6d-4bb4-acc9-12c79edaa59e None None] Validating token access rules against request {{(pid=80060) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:08:25.585600 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931379a5-ab8f-406d-87a0-d9a434b5b2cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512c115af57c463eb1fbe636667306dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46e5f537a906431ba5baa049a20664df', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80060) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:08:25.586228 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931379a5-ab8f-406d-87a0-d9a434b5b2cf service nova] REQUEST_METHOD: `GET` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:08:25.586509 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931379a5-ab8f-406d-87a0-d9a434b5b2cf service nova] SCRIPT_NAME: `/identity` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:08:25.586783 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931379a5-ab8f-406d-87a0-d9a434b5b2cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80060) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:08:25.595260 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931379a5-ab8f-406d-87a0-d9a434b5b2cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:08:25.596915 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931379a5-ab8f-406d-87a0-d9a434b5b2cf service nova] RBAC: Authorization granted {{(pid=80060) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:08:25.606238 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: [pid: 80060|app: 0|req: 676/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:08:47.620273 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: Mon May 9 15:08:47 2022 - graceful shutdown triggered... May 09 15:08:47.620748 dv-cinder-840678-3-iscsi devstack@keystone.service[80060]: Gracefully killing worker 1 (pid: 80060)... May 09 15:08:47.621145 dv-cinder-840678-3-iscsi devstack@keystone.service[80062]: Gracefully killing worker 2 (pid: 80062)... May 09 15:08:47.622990 dv-cinder-840678-3-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 09 15:08:48.621409 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: worker 1 buried after 1 seconds May 09 15:08:48.622196 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: worker 2 buried after 1 seconds May 09 15:08:48.622196 dv-cinder-840678-3-iscsi devstack@keystone.service[80049]: goodbye to uWSGI. May 09 15:08:48.623217 dv-cinder-840678-3-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 09 15:08:48.623756 dv-cinder-840678-3-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 09 15:14:09.604243 dv-cinder-840678-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 09 15:14:09.611845 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 9 15:14:09 2022] *** May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: nodename: dv-cinder-840678-3-iscsi May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: machine: x86_64 May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: clock source: unix May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: pcre jit disabled May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: detected number of CPU cores: 4 May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: current working directory: / May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: detected binary path: /usr/bin/uwsgi-core May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: your processes number limit is 47680 May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: your memory page size is 4096 bytes May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: detected max file descriptor number: 2048 May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: lock engine: pthread robust mutexes May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: thunder lock: enabled May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 09 15:14:09.615655 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 09 15:14:09.662949 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: Python main interpreter initialized at 0x562dc19ce9c0 May 09 15:14:09.662949 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: python threads support enabled May 09 15:14:09.662949 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: your server socket listen backlog is limited to 100 connections May 09 15:14:09.662949 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: your mercy for graceful operations on workers is 90 seconds May 09 15:14:09.663391 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: mapped 403077 bytes (393 KB) for 2 cores May 09 15:14:09.663391 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: *** Operational MODE: preforking *** May 09 15:14:09.663654 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: *** uWSGI is running in multiple interpreter mode *** May 09 15:14:09.663654 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: spawned uWSGI master process (pid: 143100) May 09 15:14:09.663865 dv-cinder-840678-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 09 15:14:09.664466 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: spawned uWSGI worker 1 (pid: 143116, cores: 1) May 09 15:14:09.669297 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: spawned uWSGI worker 2 (pid: 143117, cores: 1) May 09 15:14:09.669297 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 09 15:14:15.443130 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:14:15.443130 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: from cryptography.utils import int_from_bytes May 09 15:14:15.444443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:14:15.444443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: from cryptography.utils import int_from_bytes May 09 15:14:15.445268 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:14:15.445268 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: from cryptography.utils import int_from_bytes May 09 15:14:15.446587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:14:15.446587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: from cryptography.utils import int_from_bytes May 09 15:14:15.492749 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 15:14:15.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 15:14:15.493166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] command line args: [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 15:14:15.493431 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 15:14:15.493605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ================================================================================ {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 15:14:15.493853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] admin_token = **** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.494025 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 15:14:15.494123 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] config_dir = [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.494224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 15:14:15.494318 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.494416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] command line args: [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 15:14:15.494516 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] config_source = [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.494614 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] debug = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.494701 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 15:14:15.494795 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.495051 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ================================================================================ {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 15:14:15.495133 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] admin_token = **** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.495605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.495605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] insecure_debug = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.496246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] config_dir = [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.496327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.496405 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.496489 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] config_source = [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.496575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.496650 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] debug = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.496838 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] list_limit = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.496971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.497149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] log_config_append = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.497263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.497388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.497508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] insecure_debug = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.497623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] log_dir = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.497623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] log_file = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.497839 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.497962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498116 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498261 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] list_limit = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] log_config_append = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498701 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498786 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] log_dir = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.498971 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499061 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] log_file = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499365 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499632 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499717 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.499947 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500308 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500490 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500665 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] notification_format = cadf {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500750 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500836 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.500932 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.501013 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.501248 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.501357 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.501495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] publish_errors = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.501585 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.501667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.501762 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] notification_format = cadf {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.501856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.502007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.502102 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.502260 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.502492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.502588 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.502717 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] publish_errors = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.502806 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] standard_threads = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.502930 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] strict_password_check = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503217 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503432 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] use_eventlog = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] use_journal = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] use_json = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503814 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.503908 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] use_stderr = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.504000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] use_syslog = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.504086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] standard_threads = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.504172 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] watch_log_file = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.504262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] strict_password_check = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.504417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.504516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.504610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] use_eventlog = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.504705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.504801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] use_journal = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.504891 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.504972 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] use_json = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.505106 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] use_stderr = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.505195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.505276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] use_syslog = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.505425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.505543 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] watch_log_file = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:14:15.505626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.505752 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.505854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.505994 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.506082 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.506255 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.506350 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.506499 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.506601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.506744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.506840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.506976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.507079 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.507222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.507326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.507453 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.507551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.507695 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.507785 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.507937 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508031 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508169 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508502 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508655 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508754 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.508993 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.509144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.509244 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.509393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.509496 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.509641 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.509746 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.509888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.509975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.510133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.510221 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.510362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.510463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.510591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.510685 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.510818 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.510931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.511083 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.511181 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.511329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.511408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.511572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.511662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.511798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.511890 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.512024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.512102 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.512313 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.512403 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.512480 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.512557 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.512713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.512798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.513929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.514040 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.514114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.515691 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.515795 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.515882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.515976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.516059 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.516150 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.516247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.516342 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.517507 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.517607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.517713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.517811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.517894 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.517973 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518265 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518642 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518722 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518818 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518916 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.518999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519253 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519434 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519613 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519793 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.519961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520063 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520483 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520653 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520751 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.520939 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521030 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521243 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521331 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521431 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521609 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521878 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.521959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522039 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522215 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522308 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522399 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522561 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522868 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.522985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523143 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523338 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.enabled = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.523956 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524040 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524296 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524377 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524549 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524724 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.524905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.enabled = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525009 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525459 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525652 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.525927 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526116 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526206 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526534 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526722 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.526990 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527174 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527255 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527598 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.527977 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528075 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528168 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528338 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528427 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528524 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528607 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528793 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] auth.external = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.528965 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] auth.mapped = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529159 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529253 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529350 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529544 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] auth.password = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529724 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529820 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] auth.token = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.529994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] catalog.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530275 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] auth.external = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] auth.mapped = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530527 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530697 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530778 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.530906 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531020 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] auth.password = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] auth.token = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531366 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] credential.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531456 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531610 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] credential.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] catalog.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.531952 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.532096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.532204 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.532327 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.532496 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.532594 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.532677 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.532784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.532877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.533051 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] credential.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.533206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.533301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.533382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] credential.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.533524 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.533626 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.533773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.533884 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534123 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534267 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534369 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534749 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534837 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.534995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.535085 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.535228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] federation.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.535327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.535460 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.535558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.535703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] federation.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.535943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536043 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536413 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536647 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536759 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.536986 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] federation.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.537152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.537243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.537405 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.537512 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] federation.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.537639 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.537744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.537871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.537968 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.538101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.538199 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.538346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.538434 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.538588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.538679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.538821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.538912 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.539055 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.539156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.539296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.539385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.539522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.539612 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.539755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.539859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540246 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.540931 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.541022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.541195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.541285 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.541434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.541535 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.541674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.541775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.541925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.542016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.542691 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.542801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.542930 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543030 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543270 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543406 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543494 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543634 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543725 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543867 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.543971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.544134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.544232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.544340 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.544449 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.544540 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.544628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.544762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.544854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545171 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545269 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545782 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545875 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.545970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546065 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546148 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546391 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546494 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546851 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.546959 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.547076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.password = **** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.547167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.547310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.547400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.547542 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.547643 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.547785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.547874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548018 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548120 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548350 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548497 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548738 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.password = **** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.548966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.549097 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.549218 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.549316 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.549465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.549565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.549703 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.549805 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.549934 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550025 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550669 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550753 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.550929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.551093 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.551194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.551321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.551413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.551563 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.551660 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.551798 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.551888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552038 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552597 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552723 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552821 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.552953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.553089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.553206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.553293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.553422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.553523 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.553664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.553764 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.553905 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554147 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554245 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554479 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554847 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.554937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.555079 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.555168 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.555308 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.555399 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.555538 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.555630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.555770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.555872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556034 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556313 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556459 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.556938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] policy.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.557065 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.557188 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.557279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.557427 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.557531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.557661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.558480 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.558730 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.558830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] receipt.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.558974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.559064 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.559239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] policy.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.559342 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.559472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.559572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.559704 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.559793 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.559944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560035 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] receipt.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560285 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] resource.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560419 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] resource.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560749 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.560996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.561178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.561263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.561377 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.561474 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.561618 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] resource.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.561723 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] revoke.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.561872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.561962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.562106 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.562204 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] resource.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.562343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] role.cache_time = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.562444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.562577 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] role.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.562667 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.562819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] role.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.562909 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.563048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] role.list_limit = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.563148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] revoke.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.563294 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.563384 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.563533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.563636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.563774 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.563886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] role.cache_time = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564009 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] role.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564098 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564250 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] role.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564480 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] role.list_limit = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564722 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.564965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.565075 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.565218 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.565311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.565462 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.565553 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.565704 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.565803 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.565947 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.566038 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.566189 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.566281 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.566437 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.566584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.566691 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.566821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.566919 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.567067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.567158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.567311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.567429 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.567556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.567647 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.567806 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568028 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568117 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568361 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568603 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568750 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568841 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.568979 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.569132 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.569219 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.569300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.569449 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.569554 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.569696 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.569800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.569948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570038 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570424 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570655 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570882 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] token.cache_time = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.570991 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.571114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] token.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.571205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.571347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.571439 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.571589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] token.provider = fernet {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.571693 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.571821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.571914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.572059 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.572156 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.572294 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] token.cache_time = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.572394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.572526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.572618 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] token.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.572763 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.572851 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.573000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.573121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] token.provider = fernet {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.573264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.573408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] trust.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.573542 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.573633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.573780 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.573980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.574071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.574232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.574323 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.574475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.574565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.574703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.574795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.574942 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.575033 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] trust.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.575200 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.575310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.575414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.575499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.connection = **** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.575650 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.575750 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.575901 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576007 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576610 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576701 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.576940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.577103 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.577212 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.connection = **** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.577352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.577453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.577595 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.577765 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.577869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.577967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578279 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578613 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578753 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578843 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.578997 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.579100 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.579222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.579325 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.579460 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.579560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.579690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.579798 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.579883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=143116) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 15:14:15.579976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.580166 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.580405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.580665 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.580901 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.581185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.581429 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:14:15.581601 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=143117) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 15:14:15.610886 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.611108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.611336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.612097 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.612334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.612560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.620072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.622219 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.648931 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.649183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.650415 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.650651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.701951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.702731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.753484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.753659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.754517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.754756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.820077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.820313 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.820405 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.820538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.820648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.820761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.820854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.820983 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.821097 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.821227 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.821340 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.821476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.844530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.844763 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.844995 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845344 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845470 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845756 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.845992 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.846108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.846324 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.857699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.858189 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.872163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.872286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.872398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.872514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.872623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.872724 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.872842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.872943 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.873056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.873162 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.873279 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.873390 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.873476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=143116) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.873559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=143117) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:14:15.890813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.891020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:14:15.891130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.891288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:14:15.891394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.891540 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:14:15.891633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.891804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.891897 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:14:15.892058 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.892151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.892312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.892407 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.892564 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.892656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.892819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.904100 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.904370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.904635 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.904753 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.904915 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.905032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.905312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.905593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.912972 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.913799 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.916241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.916514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.916899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.917719 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.917827 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.917940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.918046 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.918252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.918346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.918516 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.918608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.918786 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.918880 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.919049 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.919141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.919313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.919405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.919573 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.919667 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.919910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.920192 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.920460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.945924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.946922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.947530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.947804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.948573 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.948853 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.953108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.953384 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.953659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.954147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.954430 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.954709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.961406 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.961682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.962038 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.966815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.967098 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.967376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.967496 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.967653 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.967783 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.967922 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.968065 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.968188 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.968347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.968453 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.968623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.968715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.968898 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.969195 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.969468 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.987812 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.988085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.988851 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.989155 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.992573 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.993724 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:15.996469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.996758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.997982 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:15.998280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.001712 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.003304 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.005311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.006983 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.008960 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.009276 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.009558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.009840 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.010116 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.010392 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.010774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.011098 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.011383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.011682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.011963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.012243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.029812 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.030181 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.030446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.030706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.031780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.032154 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.032425 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.032694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.038599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.040307 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.040575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.040681 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.042910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.043184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.044221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.044497 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.044774 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.045065 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.045338 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.046288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.046570 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.046859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.047140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.047419 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.058737 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.059014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.059291 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.059575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.059847 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.060123 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.060499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.060909 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.061219 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.061501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.061793 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.062084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.062377 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.062653 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.079190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.081253 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.082479 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.084544 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.085799 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.087377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.087900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.089060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.089843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.090125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.090218 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.091784 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.092061 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.092356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.097583 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.099179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.100116 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.101769 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.102430 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.102713 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.102997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.103268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.105124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.105351 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.105637 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.105914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.112437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.112733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.113007 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.115163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.115456 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.115748 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.121267 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=143116) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.121549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.121823 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.122100 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.122368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.124127 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=143117) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:14:16.124407 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.124687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.124962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.125263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.125553 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.125828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.126104 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.126556 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.126645 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.126882 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.127072 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.127163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.127438 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.127590 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.127716 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.127993 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.128094 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.128278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.128496 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.128597 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.128811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=143116) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.129119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=143117) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:14:16.169301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562dc19ce9c0 pid: 143116 (default app) May 09 15:14:16.170508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562dc19ce9c0 pid: 143117 (default app) May 09 15:14:16.172685 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854e47a-561e-4ee1-82e1-93c3f0f6dc8f None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:16.172985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854e47a-561e-4ee1-82e1-93c3f0f6dc8f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:16.173293 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854e47a-561e-4ee1-82e1-93c3f0f6dc8f None None] PATH_INFO: `/v3` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:16.177481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon May 9 15:14:16 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:24.107836 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80be46-2ad3-40ce-84f1-1f06f0481593 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:24.108445 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80be46-2ad3-40ce-84f1-1f06f0481593 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:24.108445 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80be46-2ad3-40ce-84f1-1f06f0481593 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:24.112718 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:24.118142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:24.118586 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:24.118711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:24.149173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG oslo_db.sqlalchemy.engines [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=143116) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 15:14:24.331033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG passlib.handlers.bcrypt [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=143116) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 15:14:24.331445 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG passlib.handlers.bcrypt [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=143116) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 15:14:24.386855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:24.388364 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:24.516787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 09 15:14:24.539956 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80be46-2ad3-40ce-84f1-1f06f0481593 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:24.574495 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG oslo_db.sqlalchemy.engines [None req-0c80be46-2ad3-40ce-84f1-1f06f0481593 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=143117) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 15:14:24.701859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80be46-2ad3-40ce-84f1-1f06f0481593 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:24.706146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607007bb-d3d4-46b9-a8df-4874a47f3b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f91da8032e14759941fd956b26044bc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:24.706804 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607007bb-d3d4-46b9-a8df-4874a47f3b83 admin admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:24.707084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607007bb-d3d4-46b9-a8df-4874a47f3b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:24.707358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607007bb-d3d4-46b9-a8df-4874a47f3b83 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:24.715881 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607007bb-d3d4-46b9-a8df-4874a47f3b83 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:24.766740 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: WARNING keystone.common.rbac_enforcer.enforcer [None req-607007bb-d3d4-46b9-a8df-4874a47f3b83 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 15:14:24.767076 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607007bb-d3d4-46b9-a8df-4874a47f3b83 admin admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:24.776776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 9 15:14:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 09 15:14:24.782674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:24.797405 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6f650-b218-4505-88d6-0148d2e60a9a None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:24.801908 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a4b5f-2efa-42ca-b60b-ee01a454f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f91da8032e14759941fd956b26044bc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:24.803033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4b5f-2efa-42ca-b60b-ee01a454f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:24.803325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4b5f-2efa-42ca-b60b-ee01a454f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:24.803613 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a4b5f-2efa-42ca-b60b-ee01a454f0e1 admin admin] PATH_INFO: `/v3/projects/2f91da8032e14759941fd956b26044bc` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:24.805203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a4b5f-2efa-42ca-b60b-ee01a454f0e1 admin admin] RBAC: Authorizing `identity:get_project(project_id=2f91da8032e14759941fd956b26044bc)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:24.855225 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2d0a4b5f-2efa-42ca-b60b-ee01a454f0e1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 15:14:24.855571 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a4b5f-2efa-42ca-b60b-ee01a454f0e1 admin admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:24.860131 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon May 9 15:14:24 2022] GET /identity/v3/projects/2f91da8032e14759941fd956b26044bc => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:27.100894 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936afe0b-58ea-4ce2-8e07-0b8d376a0c22 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:27.101504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936afe0b-58ea-4ce2-8e07-0b8d376a0c22 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:27.101504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936afe0b-58ea-4ce2-8e07-0b8d376a0c22 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:27.106149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:27.111553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e87f8-44e8-4349-8ef8-d0064c01fb47 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:27.111981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e87f8-44e8-4349-8ef8-d0064c01fb47 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:27.112117 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e87f8-44e8-4349-8ef8-d0064c01fb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:27.132143 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-f10e87f8-44e8-4349-8ef8-d0064c01fb47 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:27.133717 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-f10e87f8-44e8-4349-8ef8-d0064c01fb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:27.141577 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 09 15:14:27.163738 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936afe0b-58ea-4ce2-8e07-0b8d376a0c22 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:27.176122 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936afe0b-58ea-4ce2-8e07-0b8d376a0c22 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:27.180277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b4432-9163-447b-abca-332872ade190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f91da8032e14759941fd956b26044bc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:27.180897 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b4432-9163-447b-abca-332872ade190 admin admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:27.181209 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b4432-9163-447b-abca-332872ade190 admin admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:27.181485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b4432-9163-447b-abca-332872ade190 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:27.188938 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b4432-9163-447b-abca-332872ade190 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:27.190731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b4432-9163-447b-abca-332872ade190 admin admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:27.199271 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 9 15:14:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 15:14:27.204281 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e87f8-44e8-4349-8ef8-d0064c01fb47 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:27.212987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e87f8-44e8-4349-8ef8-d0064c01fb47 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:27.217899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7915f84-74bd-4e1d-a69b-12c287caeca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f91da8032e14759941fd956b26044bc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:27.218775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915f84-74bd-4e1d-a69b-12c287caeca6 admin admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:27.219056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915f84-74bd-4e1d-a69b-12c287caeca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:27.219328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7915f84-74bd-4e1d-a69b-12c287caeca6 admin admin] PATH_INFO: `/v3/users/de3600fcd96a4de8ae9f9640630b9125` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:27.220727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7915f84-74bd-4e1d-a69b-12c287caeca6 admin admin] RBAC: Authorizing `identity:get_user(user_id=de3600fcd96a4de8ae9f9640630b9125)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:27.222569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7915f84-74bd-4e1d-a69b-12c287caeca6 admin admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:27.224122 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon May 9 15:14:27 2022] GET /identity/v3/users/de3600fcd96a4de8ae9f9640630b9125 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:29.528385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6afdb-f204-4a37-ab1a-9baa4fe4b48a None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:29.528904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6afdb-f204-4a37-ab1a-9baa4fe4b48a None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:29.528904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6afdb-f204-4a37-ab1a-9baa4fe4b48a None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:29.530538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:14:29.535748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605973c-c3a9-4c67-bfd0-217c7fae2de8 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:29.536165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605973c-c3a9-4c67-bfd0-217c7fae2de8 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:29.536375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605973c-c3a9-4c67-bfd0-217c7fae2de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:29.658844 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140884e-5a23-4886-8bba-3de3ef8b62a1 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:29.659129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140884e-5a23-4886-8bba-3de3ef8b62a1 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:29.659410 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140884e-5a23-4886-8bba-3de3ef8b62a1 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:29.660768 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:14:29.661319 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-5605973c-c3a9-4c67-bfd0-217c7fae2de8 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:29.662524 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-5605973c-c3a9-4c67-bfd0-217c7fae2de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:29.666397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:29.667445 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:29.667731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:29.707131 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG passlib.handlers.bcrypt [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=143117) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 15:14:29.707557 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG passlib.handlers.bcrypt [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=143117) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 15:14:29.741285 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:29.754986 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:29.756579 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:29.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605973c-c3a9-4c67-bfd0-217c7fae2de8 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:29.854954 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:29.861415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605973c-c3a9-4c67-bfd0-217c7fae2de8 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:29.876831 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:29.894111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c7077-49db-4dd0-831d-3713d2a3279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:29.894907 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c7077-49db-4dd0-831d-3713d2a3279d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:29.895189 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c7077-49db-4dd0-831d-3713d2a3279d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:29.895461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c7077-49db-4dd0-831d-3713d2a3279d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:29.906863 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c7077-49db-4dd0-831d-3713d2a3279d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-4e6c7077-49db-4dd0-831d-3713d2a3279d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:29.918059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:29.919144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:29.920105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:14:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:14:29.924925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c7077-49db-4dd0-831d-3713d2a3279d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:29.972446 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22704cd-d109-43bf-ac0e-0a1082e2babc None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:30.004903 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371d578-5de2-4dee-bd93-c75a1150e0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:30.005926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371d578-5de2-4dee-bd93-c75a1150e0bb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:30.006218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371d578-5de2-4dee-bd93-c75a1150e0bb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:30.006500 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371d578-5de2-4dee-bd93-c75a1150e0bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:30.018803 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c7077-49db-4dd0-831d-3713d2a3279d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:30.050125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:30.050781 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:30.051069 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:30.051348 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:30.052099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:30.052409 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:30.053822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:30.066765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:30.070623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371d578-5de2-4dee-bd93-c75a1150e0bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:30.071051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:30.072144 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371d578-5de2-4dee-bd93-c75a1150e0bb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:30.164432 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0d299-0426-4ad8-91d7-7889959badae None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:30.164652 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:14:30.170735 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371d578-5de2-4dee-bd93-c75a1150e0bb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:30.203550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11ec23-8b4a-46ad-b81b-2758d0622cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:30.203550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ec23-8b4a-46ad-b81b-2758d0622cab None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:30.203550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ec23-8b4a-46ad-b81b-2758d0622cab None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:30.203550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11ec23-8b4a-46ad-b81b-2758d0622cab None admin] PATH_INFO: `/v3/users/de3600fcd96a4de8ae9f9640630b9125` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:30.204014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11ec23-8b4a-46ad-b81b-2758d0622cab None admin] RBAC: Authorizing `identity:get_user(user_id=de3600fcd96a4de8ae9f9640630b9125)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:30.205167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11ec23-8b4a-46ad-b81b-2758d0622cab None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:30.207040 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:30 2022] GET /identity/v3/users/de3600fcd96a4de8ae9f9640630b9125 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:30.212108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11ec23-8b4a-46ad-b81b-2758d0622cab None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:30.274716 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371d578-5de2-4dee-bd93-c75a1150e0bb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:30.306529 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11ec23-8b4a-46ad-b81b-2758d0622cab None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:30.306867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:30.307689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:30.307977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:30.308258 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:30.341066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a965e-5034-47d4-a874-e5f308ca79b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:30.341889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a965e-5034-47d4-a874-e5f308ca79b3 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:30.342174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a965e-5034-47d4-a874-e5f308ca79b3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:30.342446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a965e-5034-47d4-a874-e5f308ca79b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:30.343752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a965e-5034-47d4-a874-e5f308ca79b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:30.345253 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a965e-5034-47d4-a874-e5f308ca79b3 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:30.346947 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:30.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a965e-5034-47d4-a874-e5f308ca79b3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:30.434777 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:30.436169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:30.451797 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a965e-5034-47d4-a874-e5f308ca79b3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:14:30.457541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:30.458679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:30.459627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:14:30.459627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:14:30.459627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:14:30.459627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:30.459627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:14:30.465361 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:30.485487 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18162aa-805d-46a0-a7a0-243ba9fcd99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:30.486137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18162aa-805d-46a0-a7a0-243ba9fcd99a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:30.486423 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18162aa-805d-46a0-a7a0-243ba9fcd99a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:30.486693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18162aa-805d-46a0-a7a0-243ba9fcd99a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:30.487518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18162aa-805d-46a0-a7a0-243ba9fcd99a None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.domain.id=default, user.id=de3600fcd96a4de8ae9f9640630b9125) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:30.487826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18162aa-805d-46a0-a7a0-243ba9fcd99a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:30.489286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18162aa-805d-46a0-a7a0-243ba9fcd99a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:30.496093 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 9 15:14:30 2022] GET /identity/v3/role_assignments?user.id=de3600fcd96a4de8ae9f9640630b9125&scope.domain.id=default&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:30.567519 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f598a59-b816-4df0-b5d4-0bda862077aa None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:30.602976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48165a-ea65-4cc0-b65c-c436a2e001bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:30.603656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48165a-ea65-4cc0-b65c-c436a2e001bd None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:30.603943 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48165a-ea65-4cc0-b65c-c436a2e001bd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:30.604255 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48165a-ea65-4cc0-b65c-c436a2e001bd None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:30.605005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48165a-ea65-4cc0-b65c-c436a2e001bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:30.605347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48165a-ea65-4cc0-b65c-c436a2e001bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:30.606778 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48165a-ea65-4cc0-b65c-c436a2e001bd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:30.625634 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:33.212737 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b21fa7-44ce-4394-9a33-8ef16b0030d9 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.214326 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b21fa7-44ce-4394-9a33-8ef16b0030d9 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.214859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b21fa7-44ce-4394-9a33-8ef16b0030d9 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.217628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4638-aa04-40e6-9616-4ad55200345c None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.218441 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4638-aa04-40e6-9616-4ad55200345c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.218947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4638-aa04-40e6-9616-4ad55200345c None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.220540 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:14:33.221776 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 12/23] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:33.226254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee28fe-ceeb-4775-868d-3d7c850bae88 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.226840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee28fe-ceeb-4775-868d-3d7c850bae88 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.227332 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee28fe-ceeb-4775-868d-3d7c850bae88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.234703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415602c6-ab87-4b4d-9683-948f6b08c477 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.235275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415602c6-ab87-4b4d-9683-948f6b08c477 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.235761 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415602c6-ab87-4b4d-9683-948f6b08c477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.252190 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-94ee28fe-ceeb-4775-868d-3d7c850bae88 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:33.255781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-94ee28fe-ceeb-4775-868d-3d7c850bae88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:33.290104 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-415602c6-ab87-4b4d-9683-948f6b08c477 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:33.291322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-415602c6-ab87-4b4d-9683-948f6b08c477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:33.348293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 12/24] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:33.395867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee28fe-ceeb-4775-868d-3d7c850bae88 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:33.422329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 09 15:14:33.462227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415602c6-ab87-4b4d-9683-948f6b08c477 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:33.557333 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee28fe-ceeb-4775-868d-3d7c850bae88 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:33.577415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415602c6-ab87-4b4d-9683-948f6b08c477 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:33.612703 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2fc0b-eb2d-4678-8caf-4147525bb476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:33.613549 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2fc0b-eb2d-4678-8caf-4147525bb476 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.613832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2fc0b-eb2d-4678-8caf-4147525bb476 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.614110 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2fc0b-eb2d-4678-8caf-4147525bb476 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.617485 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:33.618281 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.618564 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.618879 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.634729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:33.636760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:33.643964 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2fc0b-eb2d-4678-8caf-4147525bb476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-76e2fc0b-eb2d-4678-8caf-4147525bb476 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:33.653505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:33.654745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:33.655729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:33.657736 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 13/26] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:14:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:33.659134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df1cd6-60e8-47aa-90cb-ab44ee32032c None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.659649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df1cd6-60e8-47aa-90cb-ab44ee32032c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.660146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df1cd6-60e8-47aa-90cb-ab44ee32032c None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.661982 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 14/27] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:14:33.668078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:33.669031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:33.670268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:14:33.670268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:14:33.670268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:14:33.670268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:33.670509 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a6311-016e-4ec2-aefe-fb672519b564 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.671072 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a6311-016e-4ec2-aefe-fb672519b564 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.671568 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a6311-016e-4ec2-aefe-fb672519b564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.672588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 14/28] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:14:33.673748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:33.714180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-1e6a6311-016e-4ec2-aefe-fb672519b564 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:33.715508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-1e6a6311-016e-4ec2-aefe-fb672519b564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:33.784046 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a811cc-11ce-474f-bc2e-c407a251d76e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:33.789682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 15/29] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:33.790839 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a6311-016e-4ec2-aefe-fb672519b564 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:33.842686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7e300-4c62-4dec-bc5b-6ea920de0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:33.844481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7e300-4c62-4dec-bc5b-6ea920de0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.845442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7e300-4c62-4dec-bc5b-6ea920de0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.845980 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7e300-4c62-4dec-bc5b-6ea920de0cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.846941 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7e300-4c62-4dec-bc5b-6ea920de0cd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:33.847562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7e300-4c62-4dec-bc5b-6ea920de0cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:33.849522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7e300-4c62-4dec-bc5b-6ea920de0cd2 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:33.865196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 15/30] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:33.865196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d214f-0088-4835-811e-a2dd1c376855 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.865196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d214f-0088-4835-811e-a2dd1c376855 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.865196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d214f-0088-4835-811e-a2dd1c376855 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.865196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 16/31] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:33.865196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d214f-0088-4835-811e-a2dd1c376855 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:33.910453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a6311-016e-4ec2-aefe-fb672519b564 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:33.945354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:33.946365 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:33.946847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:33.947333 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:33.948861 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:33.949372 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:33.950982 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:33.964725 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 16/32] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:33.965926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:33.978703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d214f-0088-4835-811e-a2dd1c376855 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:34.013692 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427dd10-71e7-4165-873d-c54da0894fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:34.014361 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427dd10-71e7-4165-873d-c54da0894fe9 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.014644 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427dd10-71e7-4165-873d-c54da0894fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.014919 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427dd10-71e7-4165-873d-c54da0894fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.015724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427dd10-71e7-4165-873d-c54da0894fe9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:34.017226 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9427dd10-71e7-4165-873d-c54da0894fe9 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:34.074098 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4158be-943f-4935-89f1-717b136b3c11 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:34.118346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe986c-9f2e-444f-881f-5e4b15e1c4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:34.119587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe986c-9f2e-444f-881f-5e4b15e1c4a3 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.120668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe986c-9f2e-444f-881f-5e4b15e1c4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.121214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe986c-9f2e-444f-881f-5e4b15e1c4a3 None admin] PATH_INFO: `/v3/users/de3600fcd96a4de8ae9f9640630b9125` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.123023 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe986c-9f2e-444f-881f-5e4b15e1c4a3 None admin] RBAC: Authorizing `identity:get_user(user_id=de3600fcd96a4de8ae9f9640630b9125)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:34.124742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe986c-9f2e-444f-881f-5e4b15e1c4a3 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:34.126749 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 17/33] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:33 2022] GET /identity/v3/users/de3600fcd96a4de8ae9f9640630b9125 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:34.128114 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fac41-22a1-4287-b678-94a3dcc21a35 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.128645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fac41-22a1-4287-b678-94a3dcc21a35 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.129224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fac41-22a1-4287-b678-94a3dcc21a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.147421 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-c12fac41-22a1-4287-b678-94a3dcc21a35 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:34.148923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-c12fac41-22a1-4287-b678-94a3dcc21a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:34.172379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 17/34] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:14:33 2022] POST /identity/v3/roles => generated 225 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:14:34.178190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427dd10-71e7-4165-873d-c54da0894fe9 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:34.273680 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 18/35] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:34.274483 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fac41-22a1-4287-b678-94a3dcc21a35 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:34.349386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427dd10-71e7-4165-873d-c54da0894fe9 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:34.389943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2de584-d73a-4fc0-b5cd-1bbde818831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:34.391020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2de584-d73a-4fc0-b5cd-1bbde818831c None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.391594 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2de584-d73a-4fc0-b5cd-1bbde818831c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.392099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2de584-d73a-4fc0-b5cd-1bbde818831c None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.393067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2de584-d73a-4fc0-b5cd-1bbde818831c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:34.394966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2de584-d73a-4fc0-b5cd-1bbde818831c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:34.430287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 18/36] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:14:34 2022] POST /identity/v3/projects => generated 301 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:14:34.431589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2de584-d73a-4fc0-b5cd-1bbde818831c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:34.441861 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12fac41-22a1-4287-b678-94a3dcc21a35 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:34.545843 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650302b4-1d3d-48ff-998e-8fc5ab018655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:34.546641 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650302b4-1d3d-48ff-998e-8fc5ab018655 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.546915 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650302b4-1d3d-48ff-998e-8fc5ab018655 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.547205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650302b4-1d3d-48ff-998e-8fc5ab018655 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.548474 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650302b4-1d3d-48ff-998e-8fc5ab018655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:34.549966 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650302b4-1d3d-48ff-998e-8fc5ab018655 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:34.556463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 19/37] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:34.567805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650302b4-1d3d-48ff-998e-8fc5ab018655 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:34.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2de584-d73a-4fc0-b5cd-1bbde818831c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:34.641032 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e187266-7785-41b8-8c39-7a650c166b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:34.642138 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187266-7785-41b8-8c39-7a650c166b6b None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.642640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187266-7785-41b8-8c39-7a650c166b6b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.643142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e187266-7785-41b8-8c39-7a650c166b6b None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.644153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e187266-7785-41b8-8c39-7a650c166b6b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:34.645924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e187266-7785-41b8-8c39-7a650c166b6b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:34.668751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 19/38] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:14:34 2022] POST /identity/v3/roles => generated 219 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:14:34.779033 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650302b4-1d3d-48ff-998e-8fc5ab018655 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:34.816686 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f8cb4-08ae-4b1e-ba37-6cb5f80738e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:34.817611 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f8cb4-08ae-4b1e-ba37-6cb5f80738e7 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.817888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f8cb4-08ae-4b1e-ba37-6cb5f80738e7 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.818156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f8cb4-08ae-4b1e-ba37-6cb5f80738e7 None admin] PATH_INFO: `/v3/domains/default/users/de3600fcd96a4de8ae9f9640630b9125/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.820216 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f8cb4-08ae-4b1e-ba37-6cb5f80738e7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=de3600fcd96a4de8ae9f9640630b9125, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:34.821718 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f8cb4-08ae-4b1e-ba37-6cb5f80738e7 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:34.888276 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d5452-2a44-4491-a2ab-88933ba6aa75 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.888950 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d5452-2a44-4491-a2ab-88933ba6aa75 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.889476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d5452-2a44-4491-a2ab-88933ba6aa75 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.891137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 20/39] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:34.897653 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f89b35-e5ea-41b3-968f-98ebe0eb5786 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.898436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f89b35-e5ea-41b3-968f-98ebe0eb5786 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.899052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f89b35-e5ea-41b3-968f-98ebe0eb5786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.927303 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-f6f89b35-e5ea-41b3-968f-98ebe0eb5786 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:34.929125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-f6f89b35-e5ea-41b3-968f-98ebe0eb5786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:34.952604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 20/40] 10.222.0.13 () {58 vars in 1361 bytes} [Mon May 9 15:14:34 2022] PUT /identity/v3/domains/default/users/de3600fcd96a4de8ae9f9640630b9125/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:14:34.954630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db0ab-b2d0-4989-b986-2e47ec6c534c None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.954924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db0ab-b2d0-4989-b986-2e47ec6c534c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.955202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db0ab-b2d0-4989-b986-2e47ec6c534c None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.959654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 21/41] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:34.967404 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3c6b5-e9d3-4768-b498-276a6cf74087 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:34.967692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3c6b5-e9d3-4768-b498-276a6cf74087 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:34.967978 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3c6b5-e9d3-4768-b498-276a6cf74087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:34.986635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-f2d3c6b5-e9d3-4768-b498-276a6cf74087 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:34.988537 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-f2d3c6b5-e9d3-4768-b498-276a6cf74087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:35.013145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 21/42] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:35.037275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f89b35-e5ea-41b3-968f-98ebe0eb5786 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:35.062932 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 22/43] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:35.085875 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3c6b5-e9d3-4768-b498-276a6cf74087 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:35.145535 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f89b35-e5ea-41b3-968f-98ebe0eb5786 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:35.180067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb28f2-826e-4978-adfc-9882e179104f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:35.180874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb28f2-826e-4978-adfc-9882e179104f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:35.181196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb28f2-826e-4978-adfc-9882e179104f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:35.181473 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb28f2-826e-4978-adfc-9882e179104f None admin] PATH_INFO: `/v3/domains/default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:35.183553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb28f2-826e-4978-adfc-9882e179104f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:35.185654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb28f2-826e-4978-adfc-9882e179104f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:35.187502 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 22/44] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:35.193145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb28f2-826e-4978-adfc-9882e179104f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:35.194772 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3c6b5-e9d3-4768-b498-276a6cf74087 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:35.228820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa202628-84ae-4cce-9ee2-ee2cc26210fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:35.230083 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202628-84ae-4cce-9ee2-ee2cc26210fa None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:35.230364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202628-84ae-4cce-9ee2-ee2cc26210fa None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:35.230639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa202628-84ae-4cce-9ee2-ee2cc26210fa None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:35.231391 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202628-84ae-4cce-9ee2-ee2cc26210fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:35.232914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa202628-84ae-4cce-9ee2-ee2cc26210fa None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:35.247651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 23/45] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:14:35 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:14:35.302536 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb28f2-826e-4978-adfc-9882e179104f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:35.339510 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491146d-ae32-4d15-864c-2e852aea9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:35.340198 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491146d-ae32-4d15-864c-2e852aea9550 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:35.340486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491146d-ae32-4d15-864c-2e852aea9550 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:35.340768 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491146d-ae32-4d15-864c-2e852aea9550 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:35.341541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491146d-ae32-4d15-864c-2e852aea9550 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:35.343022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491146d-ae32-4d15-864c-2e852aea9550 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:35.371272 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 23/46] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:14:35 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:14:37.223255 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e32ac5-6f61-4ca9-949a-f864799f9db4 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.223866 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e32ac5-6f61-4ca9-949a-f864799f9db4 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.223866 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e32ac5-6f61-4ca9-949a-f864799f9db4 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.225228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 24/47] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:37.230910 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf010-491c-4ac5-903a-4c09249df22c None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.231339 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf010-491c-4ac5-903a-4c09249df22c None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.231459 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf010-491c-4ac5-903a-4c09249df22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.248387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-b0bcf010-491c-4ac5-903a-4c09249df22c None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:37.249621 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-b0bcf010-491c-4ac5-903a-4c09249df22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:37.313416 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 24/48] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:37.346529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e32ac5-6f61-4ca9-949a-f864799f9db4 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:37.460082 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e32ac5-6f61-4ca9-949a-f864799f9db4 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:37.515568 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3f706-f251-40b0-8f55-fe718e7354b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:37.524774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f706-f251-40b0-8f55-fe718e7354b9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.525121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f706-f251-40b0-8f55-fe718e7354b9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.525400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f706-f251-40b0-8f55-fe718e7354b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.541555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3f706-f251-40b0-8f55-fe718e7354b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-4be3f706-f251-40b0-8f55-fe718e7354b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:37.549761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:37.550955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:37.552312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 25/49] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:14:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:37.556683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcf010-491c-4ac5-903a-4c09249df22c None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:37.631011 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b30d6-3103-40e0-bbda-f732aaee756e None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.631882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b30d6-3103-40e0-bbda-f732aaee756e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.632197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b30d6-3103-40e0-bbda-f732aaee756e None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.636369 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 26/50] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:14:37.641639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3ac3e-c9c3-48f2-ba20-b076a251b54e None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.641931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3ac3e-c9c3-48f2-ba20-b076a251b54e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.642209 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3ac3e-c9c3-48f2-ba20-b076a251b54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.652877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcf010-491c-4ac5-903a-4c09249df22c None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:37.664188 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-44c3ac3e-c9c3-48f2-ba20-b076a251b54e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:37.665455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-44c3ac3e-c9c3-48f2-ba20-b076a251b54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:37.686018 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:37.686683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.686971 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.687246 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.687997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:37.688302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:37.689726 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:37.697622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 25/51] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:37.703354 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:37.731230 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 27/52] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:37.753906 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3ac3e-c9c3-48f2-ba20-b076a251b54e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:37.798164 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df24cb2-4944-43c7-a695-9d653ad60fe9 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:37.829679 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30607e61-c46b-454c-b2e5-ec1d2433f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:37.830541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607e61-c46b-454c-b2e5-ec1d2433f6ce None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.830823 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607e61-c46b-454c-b2e5-ec1d2433f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.831096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607e61-c46b-454c-b2e5-ec1d2433f6ce None admin] PATH_INFO: `/v3/users/de3600fcd96a4de8ae9f9640630b9125` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.832291 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30607e61-c46b-454c-b2e5-ec1d2433f6ce None admin] RBAC: Authorizing `identity:get_user(user_id=de3600fcd96a4de8ae9f9640630b9125)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:37.833781 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30607e61-c46b-454c-b2e5-ec1d2433f6ce None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:37.835404 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 26/53] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:37 2022] GET /identity/v3/users/de3600fcd96a4de8ae9f9640630b9125 => generated 247 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:37.841345 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30607e61-c46b-454c-b2e5-ec1d2433f6ce None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:37.847018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3ac3e-c9c3-48f2-ba20-b076a251b54e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:37.880114 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c991b87-99f4-4b72-8343-02dbfe282693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:37.880916 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c991b87-99f4-4b72-8343-02dbfe282693 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.881226 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c991b87-99f4-4b72-8343-02dbfe282693 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.881505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c991b87-99f4-4b72-8343-02dbfe282693 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.882696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c991b87-99f4-4b72-8343-02dbfe282693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:37.884173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c991b87-99f4-4b72-8343-02dbfe282693 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:37.886369 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 28/54] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:37.891782 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c991b87-99f4-4b72-8343-02dbfe282693 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:37.939585 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30607e61-c46b-454c-b2e5-ec1d2433f6ce None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:37.971978 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520abb-d7ea-4390-b431-d145cc5d4d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:37.972768 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520abb-d7ea-4390-b431-d145cc5d4d04 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:37.973075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520abb-d7ea-4390-b431-d145cc5d4d04 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:37.973354 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00520abb-d7ea-4390-b431-d145cc5d4d04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:37.974530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520abb-d7ea-4390-b431-d145cc5d4d04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:37.975969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00520abb-d7ea-4390-b431-d145cc5d4d04 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:37.977605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 27/55] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:37.983309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520abb-d7ea-4390-b431-d145cc5d4d04 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:37.986292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c991b87-99f4-4b72-8343-02dbfe282693 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:38.018850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807a099-0462-4858-8e81-903ceac2f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:38.019510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a099-0462-4858-8e81-903ceac2f5f7 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:38.019796 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a099-0462-4858-8e81-903ceac2f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:38.020073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807a099-0462-4858-8e81-903ceac2f5f7 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:38.020786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807a099-0462-4858-8e81-903ceac2f5f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:38.022261 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807a099-0462-4858-8e81-903ceac2f5f7 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:38.048466 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 29/56] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 9 15:14:37 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:14:38.079483 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00520abb-d7ea-4390-b431-d145cc5d4d04 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:38.114160 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44698147-4f39-46c5-bd35-e0fc18a5c6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:38.114842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44698147-4f39-46c5-bd35-e0fc18a5c6b7 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:38.115127 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44698147-4f39-46c5-bd35-e0fc18a5c6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:38.115406 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44698147-4f39-46c5-bd35-e0fc18a5c6b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:38.116261 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44698147-4f39-46c5-bd35-e0fc18a5c6b7 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.domain.id=default, user.id=de3600fcd96a4de8ae9f9640630b9125) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:38.116576 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44698147-4f39-46c5-bd35-e0fc18a5c6b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:38.118098 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44698147-4f39-46c5-bd35-e0fc18a5c6b7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:38.125051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 28/57] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 9 15:14:37 2022] GET /identity/v3/role_assignments?user.id=de3600fcd96a4de8ae9f9640630b9125&scope.domain.id=default&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:40.349451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da1ad-399a-43cd-912e-28c5f1560b1a None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:40.349978 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da1ad-399a-43cd-912e-28c5f1560b1a None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:40.350092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da1ad-399a-43cd-912e-28c5f1560b1a None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:40.351391 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 30/58] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:40.357117 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b3d44-afb9-4261-a33b-ded3f09a2600 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:40.357550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b3d44-afb9-4261-a33b-ded3f09a2600 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:40.357694 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b3d44-afb9-4261-a33b-ded3f09a2600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:40.374598 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-743b3d44-afb9-4261-a33b-ded3f09a2600 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:40.375798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-743b3d44-afb9-4261-a33b-ded3f09a2600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:40.438422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 29/59] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:40.462561 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603da1ad-399a-43cd-912e-28c5f1560b1a None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:40.562113 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603da1ad-399a-43cd-912e-28c5f1560b1a None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:40.593873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81acfe-675b-4e1c-bd21-61cf5249c65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:40.594653 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81acfe-675b-4e1c-bd21-61cf5249c65b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:40.594938 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81acfe-675b-4e1c-bd21-61cf5249c65b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:40.595218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81acfe-675b-4e1c-bd21-61cf5249c65b None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:40.596397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81acfe-675b-4e1c-bd21-61cf5249c65b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:40.597897 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81acfe-675b-4e1c-bd21-61cf5249c65b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:40.599481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 31/60] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:40.605425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b3d44-afb9-4261-a33b-ded3f09a2600 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:40.697277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b3d44-afb9-4261-a33b-ded3f09a2600 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:40.729482 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e247c6c-8cf3-40ef-8296-d49fdfaeca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:40.730163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247c6c-8cf3-40ef-8296-d49fdfaeca15 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:40.730452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247c6c-8cf3-40ef-8296-d49fdfaeca15 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:40.730733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247c6c-8cf3-40ef-8296-d49fdfaeca15 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:40.731440 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e247c6c-8cf3-40ef-8296-d49fdfaeca15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:40.732871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e247c6c-8cf3-40ef-8296-d49fdfaeca15 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:40.757170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 30/61] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:14:40 2022] POST /identity/v3/users => generated 275 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:14:44.219366 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba686736-e8ba-4003-a4f3-036795a1a917 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.220453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba686736-e8ba-4003-a4f3-036795a1a917 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.220951 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba686736-e8ba-4003-a4f3-036795a1a917 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.222594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 32/62] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:44.228266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814ebdb-a993-405a-8f82-3730901a5558 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.228726 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814ebdb-a993-405a-8f82-3730901a5558 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.228857 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814ebdb-a993-405a-8f82-3730901a5558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.257221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-3814ebdb-a993-405a-8f82-3730901a5558 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:44.258442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-3814ebdb-a993-405a-8f82-3730901a5558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:44.312184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ade46e-0aac-4f0f-9358-3d4a6c5e58aa None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.312835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ade46e-0aac-4f0f-9358-3d4a6c5e58aa None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.313407 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ade46e-0aac-4f0f-9358-3d4a6c5e58aa None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.315001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 33/63] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:14:44.320215 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8f777-5ef8-48e1-83bd-a58166ea7e9b None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.320855 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8f777-5ef8-48e1-83bd-a58166ea7e9b None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.321373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8f777-5ef8-48e1-83bd-a58166ea7e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.347435 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 31/64] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:44.354121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-fed8f777-5ef8-48e1-83bd-a58166ea7e9b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:44.356090 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-fed8f777-5ef8-48e1-83bd-a58166ea7e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:44.371783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814ebdb-a993-405a-8f82-3730901a5558 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:44.449785 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 34/65] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:44.451513 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a904-5e0f-4434-b158-5e043ae4cf0c None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.451971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a904-5e0f-4434-b158-5e043ae4cf0c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.452400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a904-5e0f-4434-b158-5e043ae4cf0c None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.459313 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 35/66] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:44.460774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b085e1-72cb-49dd-af0b-da14f489b817 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.461289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b085e1-72cb-49dd-af0b-da14f489b817 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.461775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b085e1-72cb-49dd-af0b-da14f489b817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.492028 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-48b085e1-72cb-49dd-af0b-da14f489b817 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:44.493972 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-48b085e1-72cb-49dd-af0b-da14f489b817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:44.512174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814ebdb-a993-405a-8f82-3730901a5558 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:44.555167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3e444-5716-429a-a241-8cb7675f8b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:44.556149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3e444-5716-429a-a241-8cb7675f8b6e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.556424 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3e444-5716-429a-a241-8cb7675f8b6e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.556700 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3e444-5716-429a-a241-8cb7675f8b6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.569471 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 36/67] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:44.570809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b085e1-72cb-49dd-af0b-da14f489b817 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:44.582140 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3e444-5716-429a-a241-8cb7675f8b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-67a3e444-5716-429a-a241-8cb7675f8b6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:44.598986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:44.600085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:44.600961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 32/68] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:14:44.602519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3e444-5716-429a-a241-8cb7675f8b6e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:44.705084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b085e1-72cb-49dd-af0b-da14f489b817 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:44.707897 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3e444-5716-429a-a241-8cb7675f8b6e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:44.741801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fa5a5-4d94-44c4-ad46-6ad886028527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:44.743059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa5a5-4d94-44c4-ad46-6ad886028527 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.743580 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa5a5-4d94-44c4-ad46-6ad886028527 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.744135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa5a5-4d94-44c4-ad46-6ad886028527 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.748011 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1f515-936b-4e32-96de-92cfff1c599b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:44.748926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1f515-936b-4e32-96de-92cfff1c599b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.749313 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1f515-936b-4e32-96de-92cfff1c599b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.749647 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1f515-936b-4e32-96de-92cfff1c599b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.756689 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fa5a5-4d94-44c4-ad46-6ad886028527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:44.761630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1f515-936b-4e32-96de-92cfff1c599b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e76fa5a5-4d94-44c4-ad46-6ad886028527 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:44.765572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:44.766553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:44.767422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:44.769529 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 33/69] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a4c1f515-936b-4e32-96de-92cfff1c599b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:44.769787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:44.770807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:44.771781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 37/70] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:14:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:44.772755 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b0d29-8acc-4c32-bcd0-8605a9d288c2 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.773055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b0d29-8acc-4c32-bcd0-8605a9d288c2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.773339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b0d29-8acc-4c32-bcd0-8605a9d288c2 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.773803 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fa5a5-4d94-44c4-ad46-6ad886028527 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:44.774882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 38/71] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:44.784826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf231e-fbd1-4ba3-a709-11eded633a3c None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.785151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf231e-fbd1-4ba3-a709-11eded633a3c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.785249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf231e-fbd1-4ba3-a709-11eded633a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.802530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-bddf231e-fbd1-4ba3-a709-11eded633a3c None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:44.803747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-bddf231e-fbd1-4ba3-a709-11eded633a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:44.867143 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 39/72] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:44.867994 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf231e-fbd1-4ba3-a709-11eded633a3c None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:44.881221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fa5a5-4d94-44c4-ad46-6ad886028527 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:44.914096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:44.914767 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.915054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.915334 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.916086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:44.916422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:44.917847 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:44.925576 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 34/73] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:44.926347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:44.963717 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf231e-fbd1-4ba3-a709-11eded633a3c None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:44.996098 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9990c1-3c09-4189-8477-0b1f04e2bdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:44.996690 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9990c1-3c09-4189-8477-0b1f04e2bdb0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:44.996970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9990c1-3c09-4189-8477-0b1f04e2bdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:44.997287 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9990c1-3c09-4189-8477-0b1f04e2bdb0 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:44.998032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9990c1-3c09-4189-8477-0b1f04e2bdb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:44.998336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9990c1-3c09-4189-8477-0b1f04e2bdb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:44.999726 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9990c1-3c09-4189-8477-0b1f04e2bdb0 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.010984 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 40/74] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:14:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:45.012186 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56b9d-3d6b-48d2-9780-b42f55c90449 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.012528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56b9d-3d6b-48d2-9780-b42f55c90449 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.012808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56b9d-3d6b-48d2-9780-b42f55c90449 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.014062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 41/75] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:45.016125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56b9d-3d6b-48d2-9780-b42f55c90449 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.027123 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e793c01-096b-4eaa-bd12-b8dcd2c11e8d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.059015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.059683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.059973 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.060255 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.061008 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] RBAC: Adding query filter params (name=member) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:45.061352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:45.062728 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.070635 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 35/76] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:45.071339 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.118278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56b9d-3d6b-48d2-9780-b42f55c90449 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.150993 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeedc4-ff82-46ef-b721-85ba2496dad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.151780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeedc4-ff82-46ef-b721-85ba2496dad9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.152067 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeedc4-ff82-46ef-b721-85ba2496dad9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.152347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeedc4-ff82-46ef-b721-85ba2496dad9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.153569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeedc4-ff82-46ef-b721-85ba2496dad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:45.155040 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeedc4-ff82-46ef-b721-85ba2496dad9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.156657 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 42/77] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:45.158649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeedc4-ff82-46ef-b721-85ba2496dad9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.167231 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7dd3d-ef91-41eb-82c0-79258deab061 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.199858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655aedb-90df-4f40-8d83-87a7b18f71a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.200727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655aedb-90df-4f40-8d83-87a7b18f71a1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.201015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655aedb-90df-4f40-8d83-87a7b18f71a1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.201327 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655aedb-90df-4f40-8d83-87a7b18f71a1 None admin] PATH_INFO: `/v3/users/de3600fcd96a4de8ae9f9640630b9125` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.202546 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655aedb-90df-4f40-8d83-87a7b18f71a1 None admin] RBAC: Authorizing `identity:get_user(user_id=de3600fcd96a4de8ae9f9640630b9125)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:45.204000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655aedb-90df-4f40-8d83-87a7b18f71a1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.205660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 36/78] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/users/de3600fcd96a4de8ae9f9640630b9125 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:45.208008 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b30018-1d18-4f06-8170-f7884e305fb7 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.208296 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b30018-1d18-4f06-8170-f7884e305fb7 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.208574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b30018-1d18-4f06-8170-f7884e305fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.224034 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-d8b30018-1d18-4f06-8170-f7884e305fb7 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:45.225227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-d8b30018-1d18-4f06-8170-f7884e305fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:45.253646 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeedc4-ff82-46ef-b721-85ba2496dad9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.284489 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59bf35-b286-4db6-b182-5af74d5a900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.285387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59bf35-b286-4db6-b182-5af74d5a900f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.285674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59bf35-b286-4db6-b182-5af74d5a900f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.285955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59bf35-b286-4db6-b182-5af74d5a900f None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.289266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 37/79] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:45.289883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b30018-1d18-4f06-8170-f7884e305fb7 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.300501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59bf35-b286-4db6-b182-5af74d5a900f None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:45.302005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59bf35-b286-4db6-b182-5af74d5a900f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.303877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 43/80] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:45.304446 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59bf35-b286-4db6-b182-5af74d5a900f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.384609 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b30018-1d18-4f06-8170-f7884e305fb7 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.398631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59bf35-b286-4db6-b182-5af74d5a900f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.416801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a6cc-30c4-4190-bf28-2141b14c12d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.417710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a6cc-30c4-4190-bf28-2141b14c12d0 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.417992 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a6cc-30c4-4190-bf28-2141b14c12d0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.418268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a6cc-30c4-4190-bf28-2141b14c12d0 None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.419481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41a6cc-30c4-4190-bf28-2141b14c12d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:45.420919 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41a6cc-30c4-4190-bf28-2141b14c12d0 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.422575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 38/81] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:45.423335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a6cc-30c4-4190-bf28-2141b14c12d0 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.431215 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e93a0a-279c-4468-bf03-46921a5c2b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.431888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e93a0a-279c-4468-bf03-46921a5c2b07 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.432770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e93a0a-279c-4468-bf03-46921a5c2b07 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.433056 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e93a0a-279c-4468-bf03-46921a5c2b07 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.433790 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e93a0a-279c-4468-bf03-46921a5c2b07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:45.435238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e93a0a-279c-4468-bf03-46921a5c2b07 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.461516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 44/82] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:14:45 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:14:45.462339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e93a0a-279c-4468-bf03-46921a5c2b07 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.530774 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a6cc-30c4-4190-bf28-2141b14c12d0 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.568935 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90dae48-5462-4326-bea2-ab97ececa90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.569632 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90dae48-5462-4326-bea2-ab97ececa90b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.569922 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90dae48-5462-4326-bea2-ab97ececa90b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.570198 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90dae48-5462-4326-bea2-ab97ececa90b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.591723 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e93a0a-279c-4468-bf03-46921a5c2b07 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.630011 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b6a66-e284-4b5a-860d-8c0d37ad97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.630735 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b6a66-e284-4b5a-860d-8c0d37ad97de None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.631021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b6a66-e284-4b5a-860d-8c0d37ad97de None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.631298 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b6a66-e284-4b5a-860d-8c0d37ad97de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.640803 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90dae48-5462-4326-bea2-ab97ececa90b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:45.642333 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90dae48-5462-4326-bea2-ab97ececa90b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.708101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b6a66-e284-4b5a-860d-8c0d37ad97de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:45.709828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b6a66-e284-4b5a-860d-8c0d37ad97de None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:45.743795 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 39/83] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:45.744612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90dae48-5462-4326-bea2-ab97ececa90b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.830331 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 45/84] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:45.831196 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b6a66-e284-4b5a-860d-8c0d37ad97de None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.852981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90dae48-5462-4326-bea2-ab97ececa90b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.888137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf26be6-c55d-41f3-80eb-1c8b487501ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.889062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf26be6-c55d-41f3-80eb-1c8b487501ec None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.889266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf26be6-c55d-41f3-80eb-1c8b487501ec None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.889546 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf26be6-c55d-41f3-80eb-1c8b487501ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:45.900360 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf26be6-c55d-41f3-80eb-1c8b487501ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-eaf26be6-c55d-41f3-80eb-1c8b487501ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:45.908714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:45.909944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 40/85] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:45.911023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf26be6-c55d-41f3-80eb-1c8b487501ec None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:45.941739 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b6a66-e284-4b5a-860d-8c0d37ad97de None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:45.976112 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8623-07da-4c74-8b1b-0fd06e169b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:45.977024 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8623-07da-4c74-8b1b-0fd06e169b0e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:45.977503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8623-07da-4c74-8b1b-0fd06e169b0e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:45.977928 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8623-07da-4c74-8b1b-0fd06e169b0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.013083 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf26be6-c55d-41f3-80eb-1c8b487501ec None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.046397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8623-07da-4c74-8b1b-0fd06e169b0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.047875 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8623-07da-4c74-8b1b-0fd06e169b0e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.048480 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf2a69-3967-4a9b-822d-52fd28f60930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2a69-3967-4a9b-822d-52fd28f60930 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.051179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2a69-3967-4a9b-822d-52fd28f60930 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.051642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf2a69-3967-4a9b-822d-52fd28f60930 None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.053029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf2a69-3967-4a9b-822d-52fd28f60930 None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.054666 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf2a69-3967-4a9b-822d-52fd28f60930 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.057029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 41/86] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:46.058445 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf2a69-3967-4a9b-822d-52fd28f60930 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:46.163978 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf2a69-3967-4a9b-822d-52fd28f60930 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.167665 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 46/87] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:46.168864 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8623-07da-4c74-8b1b-0fd06e169b0e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:46.200512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a10c4-e0eb-43d7-8f35-65797d1bcccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.201412 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a10c4-e0eb-43d7-8f35-65797d1bcccb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.201699 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a10c4-e0eb-43d7-8f35-65797d1bcccb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.202009 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a10c4-e0eb-43d7-8f35-65797d1bcccb None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.203177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a10c4-e0eb-43d7-8f35-65797d1bcccb None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.204611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a10c4-e0eb-43d7-8f35-65797d1bcccb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.206529 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 42/88] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:46 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:46.207095 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a10c4-e0eb-43d7-8f35-65797d1bcccb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:46.277846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8623-07da-4c74-8b1b-0fd06e169b0e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.312975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.313628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.313914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.314193 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.314954 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:46.315265 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.316305 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a10c4-e0eb-43d7-8f35-65797d1bcccb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.316740 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.325743 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 47/89] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:46.326874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:46.352243 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.353329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.353965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.354666 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.355717 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=84caf63eb1634db7822fe78409be9c77, user.id=de3600fcd96a4de8ae9f9640630b9125) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:46.356267 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.357968 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.363872 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 43/90] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:46 2022] GET /identity/v3/role_assignments?user.id=de3600fcd96a4de8ae9f9640630b9125&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:46.370509 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:46.434345 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3886233-0381-4e8e-bd42-8c8357654782 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.477555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f42f1-b844-4198-ba2e-19296d5ad645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.478603 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f42f1-b844-4198-ba2e-19296d5ad645 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.478929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f42f1-b844-4198-ba2e-19296d5ad645 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.479213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f42f1-b844-4198-ba2e-19296d5ad645 None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.480633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f42f1-b844-4198-ba2e-19296d5ad645 None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.482166 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f42f1-b844-4198-ba2e-19296d5ad645 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.483972 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 48/91] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:46 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:46.489506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f42f1-b844-4198-ba2e-19296d5ad645 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:46.580038 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea31acf-9ab2-4458-b146-8e6a9a71b984 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.617685 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f42f1-b844-4198-ba2e-19296d5ad645 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.652779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71d22f-626f-4f22-a1d2-c6f322ce365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.653696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71d22f-626f-4f22-a1d2-c6f322ce365f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.653983 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71d22f-626f-4f22-a1d2-c6f322ce365f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.654323 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71d22f-626f-4f22-a1d2-c6f322ce365f None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.655621 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71d22f-626f-4f22-a1d2-c6f322ce365f None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.657146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71d22f-626f-4f22-a1d2-c6f322ce365f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.658802 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 49/92] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:46 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:46.659657 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71d22f-626f-4f22-a1d2-c6f322ce365f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:46.666434 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.667118 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.667405 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.667682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.668532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] RBAC: Adding query filter params (role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=84caf63eb1634db7822fe78409be9c77, user.id=cf20a77d54c04b289feef5d1d0955a73) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:46.668847 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.670328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.687210 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 44/93] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:46 2022] GET /identity/v3/role_assignments?user.id=cf20a77d54c04b289feef5d1d0955a73&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=c8ee1507247e4b34a213a8033c46378f => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:46.687831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:46.767668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71d22f-626f-4f22-a1d2-c6f322ce365f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.804874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.805585 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.805876 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.806158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:46.807568 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] RBAC: Adding query filter params (role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=84caf63eb1634db7822fe78409be9c77, user.id=cf20a77d54c04b289feef5d1d0955a73) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:46.807886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:46.812471 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:46.819354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 50/94] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:46 2022] GET /identity/v3/role_assignments?user.id=cf20a77d54c04b289feef5d1d0955a73&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:46.912704 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36e078-a226-42ae-899a-12ff7de3281e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:46.964723 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bdb61-bfe4-4f85-afad-0620bb91b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:46.965863 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bdb61-bfe4-4f85-afad-0620bb91b2ef None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:46.966358 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bdb61-bfe4-4f85-afad-0620bb91b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:46.966836 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bdb61-bfe4-4f85-afad-0620bb91b2ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:47.049658 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bdb61-bfe4-4f85-afad-0620bb91b2ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:47.051519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bdb61-bfe4-4f85-afad-0620bb91b2ef None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:47.188384 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 45/95] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:47.194261 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:47.316473 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47da59-015c-4480-b939-997cf3dae6df None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:47.355363 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e34bf-ce1c-43a4-9070-6ffd6c080487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:47.356239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e34bf-ce1c-43a4-9070-6ffd6c080487 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:47.356527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e34bf-ce1c-43a4-9070-6ffd6c080487 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:47.356813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e34bf-ce1c-43a4-9070-6ffd6c080487 None admin] PATH_INFO: `/v3/projects/4f42e0264e93481ea152c83b3ae53059` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:47.358068 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e34bf-ce1c-43a4-9070-6ffd6c080487 None admin] RBAC: Authorizing `identity:get_project(project_id=4f42e0264e93481ea152c83b3ae53059)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:47.359544 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e34bf-ce1c-43a4-9070-6ffd6c080487 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:47.361295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 51/96] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:47 2022] GET /identity/v3/projects/4f42e0264e93481ea152c83b3ae53059 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:47.371047 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bdb61-bfe4-4f85-afad-0620bb91b2ef None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:47.511020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bdb61-bfe4-4f85-afad-0620bb91b2ef None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:47.550449 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fe9da-e48a-4e30-a484-515303beb6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:47.551545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe9da-e48a-4e30-a484-515303beb6fb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:47.552056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe9da-e48a-4e30-a484-515303beb6fb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:47.552544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fe9da-e48a-4e30-a484-515303beb6fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:47.553608 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fe9da-e48a-4e30-a484-515303beb6fb None admin] RBAC: Adding query filter params (role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=4f42e0264e93481ea152c83b3ae53059, user.id=cf20a77d54c04b289feef5d1d0955a73) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:47.554140 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fe9da-e48a-4e30-a484-515303beb6fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:47.555811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fe9da-e48a-4e30-a484-515303beb6fb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:47.561828 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 46/97] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:47 2022] GET /identity/v3/role_assignments?user.id=cf20a77d54c04b289feef5d1d0955a73&scope.project.id=4f42e0264e93481ea152c83b3ae53059&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:48.337156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf9317-f4af-4925-b32f-c8c7d82eb774 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:48.337714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf9317-f4af-4925-b32f-c8c7d82eb774 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:48.337714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf9317-f4af-4925-b32f-c8c7d82eb774 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:48.339682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 52/98] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:48.345105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc1a6a-8798-4fe9-911d-9f5910a2080f None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:48.345654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc1a6a-8798-4fe9-911d-9f5910a2080f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:48.346139 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc1a6a-8798-4fe9-911d-9f5910a2080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:48.364169 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-dbbc1a6a-8798-4fe9-911d-9f5910a2080f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:48.365742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-dbbc1a6a-8798-4fe9-911d-9f5910a2080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:48.435062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 47/99] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:48.470270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf9317-f4af-4925-b32f-c8c7d82eb774 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:48.595786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf9317-f4af-4925-b32f-c8c7d82eb774 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:48.630531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29818b7d-38ab-4549-8bf1-f3560ecfbf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:48.631336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29818b7d-38ab-4549-8bf1-f3560ecfbf0c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:48.631622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29818b7d-38ab-4549-8bf1-f3560ecfbf0c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:48.631900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29818b7d-38ab-4549-8bf1-f3560ecfbf0c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:48.642322 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29818b7d-38ab-4549-8bf1-f3560ecfbf0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:48.647233 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e483174-2257-43e6-9b66-92ca7c9e9620 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:48.647772 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e483174-2257-43e6-9b66-92ca7c9e9620 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:48.648256 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e483174-2257-43e6-9b66-92ca7c9e9620 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:48.649904 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 48/100] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-29818b7d-38ab-4549-8bf1-f3560ecfbf0c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:48.653055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:48.654029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:48.654899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 53/101] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:48.656932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e483174-2257-43e6-9b66-92ca7c9e9620 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:48.659510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec2edd-fb42-4169-b506-57eb80cfbbb7 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:48.659803 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec2edd-fb42-4169-b506-57eb80cfbbb7 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:48.660092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec2edd-fb42-4169-b506-57eb80cfbbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:48.713589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-6cec2edd-fb42-4169-b506-57eb80cfbbb7 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:48.713589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-6cec2edd-fb42-4169-b506-57eb80cfbbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:48.803786 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e483174-2257-43e6-9b66-92ca7c9e9620 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:48.837779 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:48.839361 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:48.839868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:48.840356 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:48.841366 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] RBAC: Adding query filter params (name=reader) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:48.841890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:48.843494 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:48.851752 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 54/102] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:48.852466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 49/103] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:48.866493 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec2edd-fb42-4169-b506-57eb80cfbbb7 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:48.897715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:48.978687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec2edd-fb42-4169-b506-57eb80cfbbb7 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:49.024942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d88244-74bc-431a-9e8b-c5c50a41a0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:49.026392 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88244-74bc-431a-9e8b-c5c50a41a0d6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:49.026675 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88244-74bc-431a-9e8b-c5c50a41a0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:49.026948 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88244-74bc-431a-9e8b-c5c50a41a0d6 None admin] PATH_INFO: `/v3/users/48709a2b22dd472fa3736d7bbf317557` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:49.056886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d88244-74bc-431a-9e8b-c5c50a41a0d6 None admin] RBAC: Authorizing `identity:get_user(user_id=48709a2b22dd472fa3736d7bbf317557)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:49.058402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d88244-74bc-431a-9e8b-c5c50a41a0d6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:49.062123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 55/104] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:48 2022] GET /identity/v3/users/48709a2b22dd472fa3736d7bbf317557 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:49.070674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d88244-74bc-431a-9e8b-c5c50a41a0d6 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:49.072866 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853b0ff-cc41-4a2f-8373-fb5c21ad842f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:49.143943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe596a-9439-46c5-940a-079df9de6650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:49.144671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe596a-9439-46c5-940a-079df9de6650 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:49.144955 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe596a-9439-46c5-940a-079df9de6650 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:49.145254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe596a-9439-46c5-940a-079df9de6650 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:49.148433 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe596a-9439-46c5-940a-079df9de6650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:49.149985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe596a-9439-46c5-940a-079df9de6650 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:49.154495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 50/105] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:49.161965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe596a-9439-46c5-940a-079df9de6650 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:49.307223 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d88244-74bc-431a-9e8b-c5c50a41a0d6 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:49.387788 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe596a-9439-46c5-940a-079df9de6650 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:49.390989 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74159d51-144e-4b30-95ea-8968c9041f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:49.391659 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159d51-144e-4b30-95ea-8968c9041f04 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:49.391943 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159d51-144e-4b30-95ea-8968c9041f04 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:49.392232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159d51-144e-4b30-95ea-8968c9041f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:49.440278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0f132-e37c-4ee8-8f29-6e5dd2106213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:49.441703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0f132-e37c-4ee8-8f29-6e5dd2106213 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:49.442214 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0f132-e37c-4ee8-8f29-6e5dd2106213 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:49.442696 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c0f132-e37c-4ee8-8f29-6e5dd2106213 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:49.443611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0f132-e37c-4ee8-8f29-6e5dd2106213 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:49.445309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c0f132-e37c-4ee8-8f29-6e5dd2106213 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:49.499526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 51/106] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:14:49 2022] POST /identity/v3/projects => generated 302 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:14:49.535498 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74159d51-144e-4b30-95ea-8968c9041f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:49.536996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74159d51-144e-4b30-95ea-8968c9041f04 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:49.679685 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 56/107] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:49.685994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0f132-e37c-4ee8-8f29-6e5dd2106213 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:49.823926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c0f132-e37c-4ee8-8f29-6e5dd2106213 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:49.844211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206877-a8db-422b-9929-9a3491d6bb02 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:49.844553 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206877-a8db-422b-9929-9a3491d6bb02 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:49.844864 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206877-a8db-422b-9929-9a3491d6bb02 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:49.846442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 57/108] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:49.853817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741696-12c4-4609-b12b-2e48068898d3 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:49.854156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741696-12c4-4609-b12b-2e48068898d3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:49.854475 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741696-12c4-4609-b12b-2e48068898d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:49.879660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db8e0-a34c-4d75-b6f2-733f446c3dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:49.880559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db8e0-a34c-4d75-b6f2-733f446c3dbb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:49.880844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db8e0-a34c-4d75-b6f2-733f446c3dbb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:49.881144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db8e0-a34c-4d75-b6f2-733f446c3dbb None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:49.881396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-f7741696-12c4-4609-b12b-2e48068898d3 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:49.882614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-f7741696-12c4-4609-b12b-2e48068898d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:49.886439 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21db8e0-a34c-4d75-b6f2-733f446c3dbb None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:49.887948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21db8e0-a34c-4d75-b6f2-733f446c3dbb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:49.890162 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 52/109] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:49 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:49.895605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db8e0-a34c-4d75-b6f2-733f446c3dbb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:49.985807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 58/110] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:50.037679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741696-12c4-4609-b12b-2e48068898d3 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:50.046298 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db8e0-a34c-4d75-b6f2-733f446c3dbb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:50.115531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409e0b5-1d1b-4bbe-a526-556059f00e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:50.116115 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409e0b5-1d1b-4bbe-a526-556059f00e2e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.116401 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409e0b5-1d1b-4bbe-a526-556059f00e2e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.116675 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409e0b5-1d1b-4bbe-a526-556059f00e2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.117547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409e0b5-1d1b-4bbe-a526-556059f00e2e None admin] RBAC: Adding query filter params (role.id=f27561b987b14525a832c7d37ff8c62c, scope.project.id=84caf63eb1634db7822fe78409be9c77, user.id=48709a2b22dd472fa3736d7bbf317557) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:50.117855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409e0b5-1d1b-4bbe-a526-556059f00e2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:50.119360 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409e0b5-1d1b-4bbe-a526-556059f00e2e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:50.126426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 53/111] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:49 2022] GET /identity/v3/role_assignments?user.id=48709a2b22dd472fa3736d7bbf317557&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=f27561b987b14525a832c7d37ff8c62c => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:50.127608 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa458abc-37c2-46fc-bd4b-fc7ea8653845 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.127898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa458abc-37c2-46fc-bd4b-fc7ea8653845 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.128175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa458abc-37c2-46fc-bd4b-fc7ea8653845 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.129636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 54/112] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:50.135861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc07e-7435-4f05-a8ee-bb23d101efe0 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.136182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc07e-7435-4f05-a8ee-bb23d101efe0 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.136423 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc07e-7435-4f05-a8ee-bb23d101efe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.155358 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-07cbc07e-7435-4f05-a8ee-bb23d101efe0 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:50.155966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-07cbc07e-7435-4f05-a8ee-bb23d101efe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:50.186518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7741696-12c4-4609-b12b-2e48068898d3 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:50.232996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bf0ac-3799-4582-b5b9-a5f97fdc95b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:50.234501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bf0ac-3799-4582-b5b9-a5f97fdc95b4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.235297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bf0ac-3799-4582-b5b9-a5f97fdc95b4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.236039 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bf0ac-3799-4582-b5b9-a5f97fdc95b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.239380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 55/113] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:50.249848 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5bf0ac-3799-4582-b5b9-a5f97fdc95b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-3b5bf0ac-3799-4582-b5b9-a5f97fdc95b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:50.258582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:50.259633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:50.261125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:50.263595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 59/114] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:14:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:50.269727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc07e-7435-4f05-a8ee-bb23d101efe0 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:50.293906 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bf0ac-3799-4582-b5b9-a5f97fdc95b4 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:50.419886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bf0ac-3799-4582-b5b9-a5f97fdc95b4 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:50.460839 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a931f15-0e2c-4f8f-b941-5d0bc2486086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:50.462100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a931f15-0e2c-4f8f-b941-5d0bc2486086 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.462605 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a931f15-0e2c-4f8f-b941-5d0bc2486086 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.463174 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a931f15-0e2c-4f8f-b941-5d0bc2486086 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.476995 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a931f15-0e2c-4f8f-b941-5d0bc2486086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:50.479492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc07e-7435-4f05-a8ee-bb23d101efe0 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-7a931f15-0e2c-4f8f-b941-5d0bc2486086 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:50.487048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:50.488052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:50.488921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 60/115] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:14:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:50.505370 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a931f15-0e2c-4f8f-b941-5d0bc2486086 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:50.532212 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:50.533392 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.533919 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.534434 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.535457 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:50.535995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:50.537783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:50.547379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 56/116] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:50.553382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:50.624679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a931f15-0e2c-4f8f-b941-5d0bc2486086 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:50.658664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12a190-c840-4d2f-81a1-17c0c15a3171 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:50.695407 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d99e-e413-4009-b69e-cc9a2d58f218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:50.696762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d99e-e413-4009-b69e-cc9a2d58f218 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.697349 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d99e-e413-4009-b69e-cc9a2d58f218 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.697860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d99e-e413-4009-b69e-cc9a2d58f218 None admin] PATH_INFO: `/v3/users/de3600fcd96a4de8ae9f9640630b9125` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.699439 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2d99e-e413-4009-b69e-cc9a2d58f218 None admin] RBAC: Authorizing `identity:get_user(user_id=de3600fcd96a4de8ae9f9640630b9125)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:50.701216 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2d99e-e413-4009-b69e-cc9a2d58f218 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:50.703440 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 57/117] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:50 2022] GET /identity/v3/users/de3600fcd96a4de8ae9f9640630b9125 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:50.708946 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d99e-e413-4009-b69e-cc9a2d58f218 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:50.718669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572a8db-d851-4914-a289-da90524aa180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:50.719600 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572a8db-d851-4914-a289-da90524aa180 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.720046 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572a8db-d851-4914-a289-da90524aa180 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.720484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572a8db-d851-4914-a289-da90524aa180 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.721434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572a8db-d851-4914-a289-da90524aa180 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:50.721910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572a8db-d851-4914-a289-da90524aa180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:50.723463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572a8db-d851-4914-a289-da90524aa180 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:50.743567 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 61/118] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:14:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:50.757694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0304f6a-567f-4b5f-93b0-b4ff871222b6 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.758260 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0304f6a-567f-4b5f-93b0-b4ff871222b6 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.758700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0304f6a-567f-4b5f-93b0-b4ff871222b6 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.760532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 62/119] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:14:50.761439 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0304f6a-567f-4b5f-93b0-b4ff871222b6 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:50.833679 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d99e-e413-4009-b69e-cc9a2d58f218 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:50.878489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f305d5b-aa02-42ba-8321-6b136bda56aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:50.880217 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305d5b-aa02-42ba-8321-6b136bda56aa None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.880834 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305d5b-aa02-42ba-8321-6b136bda56aa None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.881778 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305d5b-aa02-42ba-8321-6b136bda56aa None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.883904 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f305d5b-aa02-42ba-8321-6b136bda56aa None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:50.886287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f305d5b-aa02-42ba-8321-6b136bda56aa None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:50.888765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 58/120] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:50 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:50.890498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbc8f1-f3fd-45e3-bc3f-82d5d12d42cb None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.891089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbc8f1-f3fd-45e3-bc3f-82d5d12d42cb None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.891609 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbc8f1-f3fd-45e3-bc3f-82d5d12d42cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.911382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0304f6a-567f-4b5f-93b0-b4ff871222b6 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:50.913018 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-5fbbc8f1-f3fd-45e3-bc3f-82d5d12d42cb None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:50.915344 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-5fbbc8f1-f3fd-45e3-bc3f-82d5d12d42cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:50.960167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaccfb-5bdc-400f-b188-31eaf9c8c770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:50.961447 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaccfb-5bdc-400f-b188-31eaf9c8c770 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.962097 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaccfb-5bdc-400f-b188-31eaf9c8c770 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.963923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaccfb-5bdc-400f-b188-31eaf9c8c770 None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.965431 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaccfb-5bdc-400f-b188-31eaf9c8c770 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:50.967121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaccfb-5bdc-400f-b188-31eaf9c8c770 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:50.969492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 63/121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:50 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:50.974497 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc1cbf-9ada-4ba7-ae5c-cf764cb4ae41 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:50.974789 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc1cbf-9ada-4ba7-ae5c-cf764cb4ae41 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:50.976567 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc1cbf-9ada-4ba7-ae5c-cf764cb4ae41 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:50.979179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 64/122] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:50.979734 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc1cbf-9ada-4ba7-ae5c-cf764cb4ae41 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.008729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 59/123] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:51.009680 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbc8f1-f3fd-45e3-bc3f-82d5d12d42cb None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.102289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc1cbf-9ada-4ba7-ae5c-cf764cb4ae41 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:51.127969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbc8f1-f3fd-45e3-bc3f-82d5d12d42cb None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:51.142839 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4eea23-d761-4dac-a160-4f03f112cacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:51.144440 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4eea23-d761-4dac-a160-4f03f112cacd None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.144795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4eea23-d761-4dac-a160-4f03f112cacd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.145172 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4eea23-d761-4dac-a160-4f03f112cacd None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77/users/de3600fcd96a4de8ae9f9640630b9125/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.147966 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4eea23-d761-4dac-a160-4f03f112cacd None admin] RBAC: Authorizing `identity:create_grant(project_id=84caf63eb1634db7822fe78409be9c77, user_id=de3600fcd96a4de8ae9f9640630b9125, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:51.149947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4eea23-d761-4dac-a160-4f03f112cacd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:51.168749 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 65/124] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:14:50 2022] PUT /identity/v3/projects/84caf63eb1634db7822fe78409be9c77/users/de3600fcd96a4de8ae9f9640630b9125/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:14:51.169846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170352e1-0da5-4731-ab05-aa59c741174d None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.170137 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170352e1-0da5-4731-ab05-aa59c741174d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.170416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170352e1-0da5-4731-ab05-aa59c741174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.181636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f7696-c2da-498d-a66e-489b806aaed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:51.183239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f7696-c2da-498d-a66e-489b806aaed5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.183797 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f7696-c2da-498d-a66e-489b806aaed5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.184346 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f7696-c2da-498d-a66e-489b806aaed5 None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.186122 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f7696-c2da-498d-a66e-489b806aaed5 None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:51.188314 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44f7696-c2da-498d-a66e-489b806aaed5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:51.191181 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 60/125] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:51 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:51.193700 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f7696-c2da-498d-a66e-489b806aaed5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.213812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-170352e1-0da5-4731-ab05-aa59c741174d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:51.214951 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-170352e1-0da5-4731-ab05-aa59c741174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:51.314006 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f7696-c2da-498d-a66e-489b806aaed5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:51.352986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5669a2-f2c7-44e7-8740-3543b6b610d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:51.354148 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5669a2-f2c7-44e7-8740-3543b6b610d1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.354537 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5669a2-f2c7-44e7-8740-3543b6b610d1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.354990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5669a2-f2c7-44e7-8740-3543b6b610d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.356322 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 66/126] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:51.357383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170352e1-0da5-4731-ab05-aa59c741174d None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.374963 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5669a2-f2c7-44e7-8740-3543b6b610d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-cf5669a2-f2c7-44e7-8740-3543b6b610d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:51.383186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:51.384381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:51.385926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 61/127] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:51.394268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5669a2-f2c7-44e7-8740-3543b6b610d1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.472733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170352e1-0da5-4731-ab05-aa59c741174d None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:51.515191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad51c68-fe04-466e-bf6a-2acf2e57a56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:51.516087 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad51c68-fe04-466e-bf6a-2acf2e57a56f None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.516373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad51c68-fe04-466e-bf6a-2acf2e57a56f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.516923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad51c68-fe04-466e-bf6a-2acf2e57a56f None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77/users/cf20a77d54c04b289feef5d1d0955a73/roles/c8ee1507247e4b34a213a8033c46378f` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.519712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad51c68-fe04-466e-bf6a-2acf2e57a56f None admin] RBAC: Authorizing `identity:create_grant(project_id=84caf63eb1634db7822fe78409be9c77, user_id=cf20a77d54c04b289feef5d1d0955a73, role_id=c8ee1507247e4b34a213a8033c46378f)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:51.521514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad51c68-fe04-466e-bf6a-2acf2e57a56f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:51.548203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 67/128] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:14:51 2022] PUT /identity/v3/projects/84caf63eb1634db7822fe78409be9c77/users/cf20a77d54c04b289feef5d1d0955a73/roles/c8ee1507247e4b34a213a8033c46378f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:14:51.553076 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad51c68-fe04-466e-bf6a-2acf2e57a56f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.602837 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5669a2-f2c7-44e7-8740-3543b6b610d1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:51.645157 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:51.646331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.646966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.647467 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.648454 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] RBAC: Adding query filter params (name=member) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:51.648991 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:51.651316 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:51.662292 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 62/129] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:51.667654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.675700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad51c68-fe04-466e-bf6a-2acf2e57a56f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:51.717401 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd38dcf-0506-4aa3-937f-d4635ae9f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:51.718298 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd38dcf-0506-4aa3-937f-d4635ae9f9cb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.718636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd38dcf-0506-4aa3-937f-d4635ae9f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.718912 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd38dcf-0506-4aa3-937f-d4635ae9f9cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.732602 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd38dcf-0506-4aa3-937f-d4635ae9f9cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-edd38dcf-0506-4aa3-937f-d4635ae9f9cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:51.741926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:51.742975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:51.743889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 68/130] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:51.750697 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd38dcf-0506-4aa3-937f-d4635ae9f9cb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.849892 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5ef48-68e6-4894-a74e-ccff34066643 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:51.858367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd38dcf-0506-4aa3-937f-d4635ae9f9cb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:51.894383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:51.895061 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.895342 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.895619 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.896415 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:51.896677 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:51.898153 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:51.907762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd7f3b-b045-4b84-a098-ab5695231a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:51.908669 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd7f3b-b045-4b84-a098-ab5695231a88 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:51.908952 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd7f3b-b045-4b84-a098-ab5695231a88 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:51.909252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd7f3b-b045-4b84-a098-ab5695231a88 None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:51.913225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 69/131] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:51.914036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd7f3b-b045-4b84-a098-ab5695231a88 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:51.915514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd7f3b-b045-4b84-a098-ab5695231a88 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:51.921575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 63/132] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:51 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:51.923450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:51.927168 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd7f3b-b045-4b84-a098-ab5695231a88 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:52.027415 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461671-9dd6-4787-ba39-1bebf3d80c2a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:52.060854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a580f-7e39-46b2-8c72-3c378c2e3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:52.061803 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a580f-7e39-46b2-8c72-3c378c2e3791 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:52.062090 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a580f-7e39-46b2-8c72-3c378c2e3791 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:52.062365 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a580f-7e39-46b2-8c72-3c378c2e3791 None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:52.063636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a580f-7e39-46b2-8c72-3c378c2e3791 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:52.065101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20a580f-7e39-46b2-8c72-3c378c2e3791 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:52.066804 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 70/133] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:51 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:52.072100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a580f-7e39-46b2-8c72-3c378c2e3791 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:52.095226 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd7f3b-b045-4b84-a098-ab5695231a88 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:52.180487 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a580f-7e39-46b2-8c72-3c378c2e3791 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:52.188498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e40b23-a7c4-4281-8ea4-a16e8fcf3b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:52.189423 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e40b23-a7c4-4281-8ea4-a16e8fcf3b8f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:52.189743 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e40b23-a7c4-4281-8ea4-a16e8fcf3b8f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:52.189986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e40b23-a7c4-4281-8ea4-a16e8fcf3b8f None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:52.193786 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e40b23-a7c4-4281-8ea4-a16e8fcf3b8f None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:52.195265 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e40b23-a7c4-4281-8ea4-a16e8fcf3b8f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:52.197198 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 64/134] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:51 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:52.203225 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e40b23-a7c4-4281-8ea4-a16e8fcf3b8f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:52.216438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a7d2-3298-4a34-ae6b-f22346d25ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:52.217392 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a7d2-3298-4a34-ae6b-f22346d25ee1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:52.217678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a7d2-3298-4a34-ae6b-f22346d25ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:52.217954 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a7d2-3298-4a34-ae6b-f22346d25ee1 None admin] PATH_INFO: `/v3/projects/4f42e0264e93481ea152c83b3ae53059` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:52.219215 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48a7d2-3298-4a34-ae6b-f22346d25ee1 None admin] RBAC: Authorizing `identity:get_project(project_id=4f42e0264e93481ea152c83b3ae53059)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:52.220735 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48a7d2-3298-4a34-ae6b-f22346d25ee1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:52.222518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 71/135] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:52 2022] GET /identity/v3/projects/4f42e0264e93481ea152c83b3ae53059 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:52.229730 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a7d2-3298-4a34-ae6b-f22346d25ee1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:52.360771 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a7d2-3298-4a34-ae6b-f22346d25ee1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:52.398300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1eaee-99f8-4718-9479-cdfff8f9dbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:52.399199 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1eaee-99f8-4718-9479-cdfff8f9dbbd None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:52.399649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1eaee-99f8-4718-9479-cdfff8f9dbbd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:52.399932 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1eaee-99f8-4718-9479-cdfff8f9dbbd None admin] PATH_INFO: `/v3/projects/4f42e0264e93481ea152c83b3ae53059/users/cf20a77d54c04b289feef5d1d0955a73/roles/77a3914adfe94a78b2474e4d74d2bf57` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:52.402648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1eaee-99f8-4718-9479-cdfff8f9dbbd None admin] RBAC: Authorizing `identity:create_grant(project_id=4f42e0264e93481ea152c83b3ae53059, user_id=cf20a77d54c04b289feef5d1d0955a73, role_id=77a3914adfe94a78b2474e4d74d2bf57)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:52.404118 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d1eaee-99f8-4718-9479-cdfff8f9dbbd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:52.424300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 72/136] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:14:52 2022] PUT /identity/v3/projects/4f42e0264e93481ea152c83b3ae53059/users/cf20a77d54c04b289feef5d1d0955a73/roles/77a3914adfe94a78b2474e4d74d2bf57 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:14:52.439817 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e40b23-a7c4-4281-8ea4-a16e8fcf3b8f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:52.477373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57493f95-8994-4ced-b6e0-0b95b1ffe4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:52.478290 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57493f95-8994-4ced-b6e0-0b95b1ffe4ff None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:52.478642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57493f95-8994-4ced-b6e0-0b95b1ffe4ff None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:52.478972 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57493f95-8994-4ced-b6e0-0b95b1ffe4ff None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77/users/cf20a77d54c04b289feef5d1d0955a73/roles/77a3914adfe94a78b2474e4d74d2bf57` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:52.481135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57493f95-8994-4ced-b6e0-0b95b1ffe4ff None admin] RBAC: Authorizing `identity:create_grant(project_id=84caf63eb1634db7822fe78409be9c77, user_id=cf20a77d54c04b289feef5d1d0955a73, role_id=77a3914adfe94a78b2474e4d74d2bf57)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:52.482610 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57493f95-8994-4ced-b6e0-0b95b1ffe4ff None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:52.508661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 65/137] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:14:52 2022] PUT /identity/v3/projects/84caf63eb1634db7822fe78409be9c77/users/cf20a77d54c04b289feef5d1d0955a73/roles/77a3914adfe94a78b2474e4d74d2bf57 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:14:53.046854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec8301-412c-453f-92b3-294ca69159c7 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:53.047405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec8301-412c-453f-92b3-294ca69159c7 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:53.047405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec8301-412c-453f-92b3-294ca69159c7 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:53.049205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 73/138] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:53.054609 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74579b2d-2ae4-4170-9587-9dafcdea296e None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:53.055035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74579b2d-2ae4-4170-9587-9dafcdea296e None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:53.055214 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74579b2d-2ae4-4170-9587-9dafcdea296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:53.076815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-74579b2d-2ae4-4170-9587-9dafcdea296e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:53.078094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-74579b2d-2ae4-4170-9587-9dafcdea296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:53.132294 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f62691-33ca-46d2-a5fa-5216228c4970 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:53.132636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f62691-33ca-46d2-a5fa-5216228c4970 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:53.132847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f62691-33ca-46d2-a5fa-5216228c4970 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:53.134551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 74/139] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:53.140038 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08167a7c-c6da-472c-9e99-a54d60c2b4a9 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:53.140355 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08167a7c-c6da-472c-9e99-a54d60c2b4a9 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:53.140600 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08167a7c-c6da-472c-9e99-a54d60c2b4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:53.164647 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-08167a7c-c6da-472c-9e99-a54d60c2b4a9 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:53.170093 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-08167a7c-c6da-472c-9e99-a54d60c2b4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:53.179584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 66/140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:53.212220 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74579b2d-2ae4-4170-9587-9dafcdea296e None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:53.289682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 75/141] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:53.334814 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08167a7c-c6da-472c-9e99-a54d60c2b4a9 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:53.447166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74579b2d-2ae4-4170-9587-9dafcdea296e None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:53.500720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08167a7c-c6da-472c-9e99-a54d60c2b4a9 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:53.537754 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e370a6-0eeb-4719-9d74-70ab7146c203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:53.538823 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e370a6-0eeb-4719-9d74-70ab7146c203 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:53.539286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e370a6-0eeb-4719-9d74-70ab7146c203 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:53.539732 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e370a6-0eeb-4719-9d74-70ab7146c203 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:53.560807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e3d8a-2319-4809-8cbd-0441423a0693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:53.562001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3d8a-2319-4809-8cbd-0441423a0693 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:53.562516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3d8a-2319-4809-8cbd-0441423a0693 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:53.563004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e3d8a-2319-4809-8cbd-0441423a0693 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:53.564533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e3d8a-2319-4809-8cbd-0441423a0693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:53.566246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e3d8a-2319-4809-8cbd-0441423a0693 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:53.568102 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 76/142] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:14:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:53.574000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e3d8a-2319-4809-8cbd-0441423a0693 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:53.575787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e370a6-0eeb-4719-9d74-70ab7146c203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b6e370a6-0eeb-4719-9d74-70ab7146c203 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:53.583456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:53.584450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:53.585466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 67/143] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:14:53.622087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e370a6-0eeb-4719-9d74-70ab7146c203 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:53.694666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e3d8a-2319-4809-8cbd-0441423a0693 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:53.732395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6f4fd-649b-411d-85b4-1f2312610b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:53.733510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6f4fd-649b-411d-85b4-1f2312610b59 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:53.734007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6f4fd-649b-411d-85b4-1f2312610b59 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:53.734510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6f4fd-649b-411d-85b4-1f2312610b59 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:53.735446 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6f4fd-649b-411d-85b4-1f2312610b59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:53.737146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6f4fd-649b-411d-85b4-1f2312610b59 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:53.762879 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 77/144] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:14:53 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:14:53.841893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e370a6-0eeb-4719-9d74-70ab7146c203 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:53.887910 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:53.888604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:53.888889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:53.889194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:53.890018 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] RBAC: Adding query filter params (name=reader) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:53.890333 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:53.891744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:53.905659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 68/145] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:53.911957 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6f4fd-649b-411d-85b4-1f2312610b59 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:54.096900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6f4fd-649b-411d-85b4-1f2312610b59 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:54.137290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8179f37-0efc-452a-a7ee-e28589ab9702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:54.138176 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179f37-0efc-452a-a7ee-e28589ab9702 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:54.138460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179f37-0efc-452a-a7ee-e28589ab9702 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:54.138738 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179f37-0efc-452a-a7ee-e28589ab9702 None admin] PATH_INFO: `/v3/users/48709a2b22dd472fa3736d7bbf317557` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:54.145784 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8179f37-0efc-452a-a7ee-e28589ab9702 None admin] RBAC: Authorizing `identity:get_user(user_id=48709a2b22dd472fa3736d7bbf317557)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:54.147260 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8179f37-0efc-452a-a7ee-e28589ab9702 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:54.149259 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 78/146] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:53 2022] GET /identity/v3/users/48709a2b22dd472fa3736d7bbf317557 => generated 289 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:54.155348 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:54.316728 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54dd6f-19e2-40e1-9835-7ceedbdd437e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:54.366614 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7f5b2-2e4e-470b-bc31-8c696e62cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:54.367414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f5b2-2e4e-470b-bc31-8c696e62cd27 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:54.367702 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f5b2-2e4e-470b-bc31-8c696e62cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:54.367982 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f5b2-2e4e-470b-bc31-8c696e62cd27 None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:54.369251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7f5b2-2e4e-470b-bc31-8c696e62cd27 None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:54.370974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7f5b2-2e4e-470b-bc31-8c696e62cd27 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:54.372994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 69/147] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:54 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:54.378813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8179f37-0efc-452a-a7ee-e28589ab9702 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:54.496029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8179f37-0efc-452a-a7ee-e28589ab9702 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:54.553745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40017cb0-37d6-4c9a-af28-f3a19d65d282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:54.554652 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40017cb0-37d6-4c9a-af28-f3a19d65d282 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:54.554937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40017cb0-37d6-4c9a-af28-f3a19d65d282 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:54.555213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40017cb0-37d6-4c9a-af28-f3a19d65d282 None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77/users/48709a2b22dd472fa3736d7bbf317557/roles/f27561b987b14525a832c7d37ff8c62c` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:54.557358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40017cb0-37d6-4c9a-af28-f3a19d65d282 None admin] RBAC: Authorizing `identity:create_grant(project_id=84caf63eb1634db7822fe78409be9c77, user_id=48709a2b22dd472fa3736d7bbf317557, role_id=f27561b987b14525a832c7d37ff8c62c)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:54.558833 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40017cb0-37d6-4c9a-af28-f3a19d65d282 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:54.579193 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 79/148] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:14:54 2022] PUT /identity/v3/projects/84caf63eb1634db7822fe78409be9c77/users/48709a2b22dd472fa3736d7bbf317557/roles/f27561b987b14525a832c7d37ff8c62c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:14:54.945295 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aae07e-6538-47c3-853f-bf499e57c45f None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:54.946039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aae07e-6538-47c3-853f-bf499e57c45f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:54.946539 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aae07e-6538-47c3-853f-bf499e57c45f None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:54.949594 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 70/149] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:54.959127 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae08e2-8a89-4529-a6c7-53c11403b7c3 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:54.959914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae08e2-8a89-4529-a6c7-53c11403b7c3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:54.960433 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae08e2-8a89-4529-a6c7-53c11403b7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:54.992089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-0fae08e2-8a89-4529-a6c7-53c11403b7c3 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:54.994013 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-0fae08e2-8a89-4529-a6c7-53c11403b7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:55.078882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 80/150] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:55.126520 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aae07e-6538-47c3-853f-bf499e57c45f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:55.245772 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aae07e-6538-47c3-853f-bf499e57c45f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:55.318850 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1d9c7-ff4e-44b8-8b17-6ff1f453d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:55.320311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1d9c7-ff4e-44b8-8b17-6ff1f453d23d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:55.320825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1d9c7-ff4e-44b8-8b17-6ff1f453d23d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:55.321353 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1d9c7-ff4e-44b8-8b17-6ff1f453d23d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:55.338094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1d9c7-ff4e-44b8-8b17-6ff1f453d23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-70d1d9c7-ff4e-44b8-8b17-6ff1f453d23d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:55.345896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:55.347569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:55.349076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 71/151] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:55.358166 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae08e2-8a89-4529-a6c7-53c11403b7c3 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:55.494438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae08e2-8a89-4529-a6c7-53c11403b7c3 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:55.535250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce26c2-be85-4de1-b78f-d6ca4dcecf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:55.536277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce26c2-be85-4de1-b78f-d6ca4dcecf38 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:55.536779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce26c2-be85-4de1-b78f-d6ca4dcecf38 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:55.537311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce26c2-be85-4de1-b78f-d6ca4dcecf38 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:55.538295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce26c2-be85-4de1-b78f-d6ca4dcecf38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:55.538806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce26c2-be85-4de1-b78f-d6ca4dcecf38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:55.540401 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce26c2-be85-4de1-b78f-d6ca4dcecf38 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:55.552579 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 81/152] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:55.560536 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1d9c7-ff4e-44b8-8b17-6ff1f453d23d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:55.635855 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a01c7-7eea-4ef5-b99d-63666353126d None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:55.636540 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a01c7-7eea-4ef5-b99d-63666353126d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:55.637065 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a01c7-7eea-4ef5-b99d-63666353126d None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:55.638783 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 82/153] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:55.646600 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9513ba-5ccb-402d-b5c1-2740b375ea82 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:55.647284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9513ba-5ccb-402d-b5c1-2740b375ea82 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:55.647780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9513ba-5ccb-402d-b5c1-2740b375ea82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:55.678873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-af9513ba-5ccb-402d-b5c1-2740b375ea82 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:55.680481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-af9513ba-5ccb-402d-b5c1-2740b375ea82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:55.711498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1d9c7-ff4e-44b8-8b17-6ff1f453d23d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:55.750108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012a1e5-f604-438a-86f1-2a87379c4eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:55.751252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a1e5-f604-438a-86f1-2a87379c4eac None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:55.751719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a1e5-f604-438a-86f1-2a87379c4eac None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:55.752208 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a1e5-f604-438a-86f1-2a87379c4eac None admin] PATH_INFO: `/v3/users/de3600fcd96a4de8ae9f9640630b9125` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:55.753693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012a1e5-f604-438a-86f1-2a87379c4eac None admin] RBAC: Authorizing `identity:get_user(user_id=de3600fcd96a4de8ae9f9640630b9125)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:55.755349 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012a1e5-f604-438a-86f1-2a87379c4eac None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:55.757511 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 72/154] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:55 2022] GET /identity/v3/users/de3600fcd96a4de8ae9f9640630b9125 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:55.759116 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4353c-a041-4d07-bccf-d528a0031d86 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:55.759581 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4353c-a041-4d07-bccf-d528a0031d86 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:55.760161 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4353c-a041-4d07-bccf-d528a0031d86 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:55.761875 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 73/155] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:55.768908 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363446-32a9-4759-a70a-0a78898081e6 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:55.769412 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363446-32a9-4759-a70a-0a78898081e6 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:55.769851 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363446-32a9-4759-a70a-0a78898081e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:55.795961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-7d363446-32a9-4759-a70a-0a78898081e6 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:55.797439 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-7d363446-32a9-4759-a70a-0a78898081e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:55.819251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 83/156] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:55.819929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9513ba-5ccb-402d-b5c1-2740b375ea82 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:55.913281 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 74/157] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:55.914506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363446-32a9-4759-a70a-0a78898081e6 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:55.985138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9513ba-5ccb-402d-b5c1-2740b375ea82 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.030143 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ed6c4-8254-42f2-b5ed-36b3a7180dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.031055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed6c4-8254-42f2-b5ed-36b3a7180dc6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.031533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed6c4-8254-42f2-b5ed-36b3a7180dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.031998 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ed6c4-8254-42f2-b5ed-36b3a7180dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.039273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363446-32a9-4759-a70a-0a78898081e6 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.079636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dcc47-74cf-419d-8bfe-98d3537e75a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.081396 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dcc47-74cf-419d-8bfe-98d3537e75a6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.081903 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dcc47-74cf-419d-8bfe-98d3537e75a6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.082344 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dcc47-74cf-419d-8bfe-98d3537e75a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.097199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dcc47-74cf-419d-8bfe-98d3537e75a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:56.101973 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ed6c4-8254-42f2-b5ed-36b3a7180dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:56.103565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ed6c4-8254-42f2-b5ed-36b3a7180dc6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b21dcc47-74cf-419d-8bfe-98d3537e75a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:56.107816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:56.109010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:56.110121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:56.112413 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 75/158] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:56.113371 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dcc47-74cf-419d-8bfe-98d3537e75a6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.212791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 84/159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:56.214403 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686680-670e-4ba1-add4-3e83eb2994e4 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.214531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686680-670e-4ba1-add4-3e83eb2994e4 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.214891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686680-670e-4ba1-add4-3e83eb2994e4 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.216295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 85/160] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:56.221101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0686680-670e-4ba1-add4-3e83eb2994e4 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.246553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dcc47-74cf-419d-8bfe-98d3537e75a6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.285370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d3ac1-716a-4946-a024-965e23b93be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.286191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3ac1-716a-4946-a024-965e23b93be2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.286475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3ac1-716a-4946-a024-965e23b93be2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.286749 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d3ac1-716a-4946-a024-965e23b93be2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.298425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d3ac1-716a-4946-a024-965e23b93be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-619d3ac1-716a-4946-a024-965e23b93be2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:56.307664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:56.308668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:56.309605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 76/161] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:56.311256 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5d72e-e7cc-4a08-8ea3-832ee5b11c89 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.311256 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5d72e-e7cc-4a08-8ea3-832ee5b11c89 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.311256 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da5d72e-e7cc-4a08-8ea3-832ee5b11c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.330261 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-1da5d72e-e7cc-4a08-8ea3-832ee5b11c89 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:56.331382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-1da5d72e-e7cc-4a08-8ea3-832ee5b11c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:56.379608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0686680-670e-4ba1-add4-3e83eb2994e4 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.404441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 77/162] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:56.405163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5d72e-e7cc-4a08-8ea3-832ee5b11c89 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.429753 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.430437 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.430722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.431003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.432317 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:56.432628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:56.434055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:56.442809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 86/163] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:56.443454 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.507615 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da5d72e-e7cc-4a08-8ea3-832ee5b11c89 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.542355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddac64-0df7-4cba-b206-c75c4e330c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.543268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddac64-0df7-4cba-b206-c75c4e330c8b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.543596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddac64-0df7-4cba-b206-c75c4e330c8b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.543915 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddac64-0df7-4cba-b206-c75c4e330c8b None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.545143 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddac64-0df7-4cba-b206-c75c4e330c8b None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:56.546590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddac64-0df7-4cba-b206-c75c4e330c8b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:56.548586 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 78/164] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:56.549395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddac64-0df7-4cba-b206-c75c4e330c8b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.564484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6ac42-64ce-469e-8ba6-a03f7b8fa921 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.600534 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.601159 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.601448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.601727 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.602490 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:56.602800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:56.606066 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:56.615400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 87/165] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:56.616044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.650698 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddac64-0df7-4cba-b206-c75c4e330c8b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.686867 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f528b1-23f7-41a2-a10f-9e87d6ca1650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.687743 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f528b1-23f7-41a2-a10f-9e87d6ca1650 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.688021 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f528b1-23f7-41a2-a10f-9e87d6ca1650 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.688291 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f528b1-23f7-41a2-a10f-9e87d6ca1650 None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.689552 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f528b1-23f7-41a2-a10f-9e87d6ca1650 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:56.691116 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f528b1-23f7-41a2-a10f-9e87d6ca1650 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:56.692752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 79/166] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:56.693556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f528b1-23f7-41a2-a10f-9e87d6ca1650 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.733522 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5dd2b-fc53-49be-a7fa-8fcc4c0f2398 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.767953 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b4e13-b7d9-4251-8ffa-758fdd4547f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.768759 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4e13-b7d9-4251-8ffa-758fdd4547f4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.769061 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4e13-b7d9-4251-8ffa-758fdd4547f4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.769364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b4e13-b7d9-4251-8ffa-758fdd4547f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.781574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b4e13-b7d9-4251-8ffa-758fdd4547f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-0d4b4e13-b7d9-4251-8ffa-758fdd4547f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:56.789417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:56.792434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 88/167] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:56.794662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b4e13-b7d9-4251-8ffa-758fdd4547f4 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.796593 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f528b1-23f7-41a2-a10f-9e87d6ca1650 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.830698 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.831423 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.831702 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.832034 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.832870 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=84caf63eb1634db7822fe78409be9c77, user.id=de3600fcd96a4de8ae9f9640630b9125) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:56.833208 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:56.834664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:56.840813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 80/168] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/role_assignments?user.id=de3600fcd96a4de8ae9f9640630b9125&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:56.841649 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:56.930124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b4e13-b7d9-4251-8ffa-758fdd4547f4 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.949977 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76713ff-0b2a-4653-8508-e2a0e81a5fb6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:56.991744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4082fd-07a1-46c6-ae25-4caa81d8fbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.992149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ead76-05f5-433f-a945-9cc6eda23919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:56.992360 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4082fd-07a1-46c6-ae25-4caa81d8fbc5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.992548 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ead76-05f5-433f-a945-9cc6eda23919 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:56.992716 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4082fd-07a1-46c6-ae25-4caa81d8fbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.992899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ead76-05f5-433f-a945-9cc6eda23919 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:56.993094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4082fd-07a1-46c6-ae25-4caa81d8fbc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.993294 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ead76-05f5-433f-a945-9cc6eda23919 None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:56.994497 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ead76-05f5-433f-a945-9cc6eda23919 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.001075 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ead76-05f5-433f-a945-9cc6eda23919 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.003481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 89/169] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:57.004600 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ead76-05f5-433f-a945-9cc6eda23919 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:57.067116 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4082fd-07a1-46c6-ae25-4caa81d8fbc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.068617 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4082fd-07a1-46c6-ae25-4caa81d8fbc5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.174660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 81/170] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:57.175599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4082fd-07a1-46c6-ae25-4caa81d8fbc5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:57.204517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ead76-05f5-433f-a945-9cc6eda23919 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:57.265759 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:57.266442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:57.266722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:57.266997 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:57.267754 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:57.268060 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.269479 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.276907 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4082fd-07a1-46c6-ae25-4caa81d8fbc5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:57.278700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 90/171] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:57.283883 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:57.311847 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44138a4-f742-4459-aa9d-da48a0ba4d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:57.312569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44138a4-f742-4459-aa9d-da48a0ba4d18 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:57.312856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44138a4-f742-4459-aa9d-da48a0ba4d18 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:57.313161 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44138a4-f742-4459-aa9d-da48a0ba4d18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:57.382432 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44138a4-f742-4459-aa9d-da48a0ba4d18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.383924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44138a4-f742-4459-aa9d-da48a0ba4d18 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.439400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba349d-4e6c-4dad-865f-fd1fe5f296e8 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:57.491212 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96428516-81b5-46b5-8f90-7ce88226c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:57.492018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428516-81b5-46b5-8f90-7ce88226c19d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:57.492294 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428516-81b5-46b5-8f90-7ce88226c19d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:57.492564 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428516-81b5-46b5-8f90-7ce88226c19d None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:57.494909 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 82/172] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:57.495756 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44138a4-f742-4459-aa9d-da48a0ba4d18 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:57.495882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96428516-81b5-46b5-8f90-7ce88226c19d None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.497329 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96428516-81b5-46b5-8f90-7ce88226c19d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.499187 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 91/173] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:57 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:57.504781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96428516-81b5-46b5-8f90-7ce88226c19d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:57.600515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44138a4-f742-4459-aa9d-da48a0ba4d18 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:57.638299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e145bb-7158-4fa1-86ae-1608dd8bf6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:57.639175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e145bb-7158-4fa1-86ae-1608dd8bf6c5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:57.639457 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e145bb-7158-4fa1-86ae-1608dd8bf6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:57.639731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e145bb-7158-4fa1-86ae-1608dd8bf6c5 None admin] PATH_INFO: `/v3/users/cf20a77d54c04b289feef5d1d0955a73` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:57.640969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e145bb-7158-4fa1-86ae-1608dd8bf6c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cf20a77d54c04b289feef5d1d0955a73)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.642481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e145bb-7158-4fa1-86ae-1608dd8bf6c5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.644116 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 83/174] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:57 2022] GET /identity/v3/users/cf20a77d54c04b289feef5d1d0955a73 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:57.644881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e145bb-7158-4fa1-86ae-1608dd8bf6c5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:57.656494 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96428516-81b5-46b5-8f90-7ce88226c19d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:57.701356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:57.702651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:57.702941 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:57.703217 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:57.709576 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] RBAC: Adding query filter params (role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=84caf63eb1634db7822fe78409be9c77, user.id=cf20a77d54c04b289feef5d1d0955a73) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:57.710085 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.712427 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.722384 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 92/175] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:57 2022] GET /identity/v3/role_assignments?user.id=cf20a77d54c04b289feef5d1d0955a73&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:57.728696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:57.758860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e145bb-7158-4fa1-86ae-1608dd8bf6c5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:57.795346 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172c454-889e-45e2-8f77-f7e9d6a0163a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:57.796222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172c454-889e-45e2-8f77-f7e9d6a0163a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:57.796504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172c454-889e-45e2-8f77-f7e9d6a0163a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:57.796778 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172c454-889e-45e2-8f77-f7e9d6a0163a None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:57.798027 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172c454-889e-45e2-8f77-f7e9d6a0163a None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.799676 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172c454-889e-45e2-8f77-f7e9d6a0163a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.801539 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 84/176] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:57 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:57.807884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172c454-889e-45e2-8f77-f7e9d6a0163a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:57.851763 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4b4f3-6a35-49a8-af4d-d971b113d739 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:57.897885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ad89d-ca99-46fe-aaf2-5a1b387be55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:57.898565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ad89d-ca99-46fe-aaf2-5a1b387be55d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:57.898846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ad89d-ca99-46fe-aaf2-5a1b387be55d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:57.899120 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ad89d-ca99-46fe-aaf2-5a1b387be55d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:57.915604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172c454-889e-45e2-8f77-f7e9d6a0163a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:57.952921 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:57.953642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:57.953923 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:57.954201 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:57.955046 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] RBAC: Adding query filter params (role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=84caf63eb1634db7822fe78409be9c77, user.id=cf20a77d54c04b289feef5d1d0955a73) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:57.955358 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:57.956820 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:57.963341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 85/177] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:57 2022] GET /identity/v3/role_assignments?user.id=cf20a77d54c04b289feef5d1d0955a73&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=c8ee1507247e4b34a213a8033c46378f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:58.000501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ad89d-ca99-46fe-aaf2-5a1b387be55d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:58.002045 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ad89d-ca99-46fe-aaf2-5a1b387be55d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:58.128134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 93/178] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:14:58.138224 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:58.273841 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce857c-7aeb-4dc6-beb7-273f7f1dd809 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:58.308906 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3534cd-05ab-4369-b724-490d30c6bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:58.309804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3534cd-05ab-4369-b724-490d30c6bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:58.310088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3534cd-05ab-4369-b724-490d30c6bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:58.310364 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3534cd-05ab-4369-b724-490d30c6bbe7 None admin] PATH_INFO: `/v3/projects/4f42e0264e93481ea152c83b3ae53059` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:58.311580 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3534cd-05ab-4369-b724-490d30c6bbe7 None admin] RBAC: Authorizing `identity:get_project(project_id=4f42e0264e93481ea152c83b3ae53059)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:58.313123 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3534cd-05ab-4369-b724-490d30c6bbe7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:58.315111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 86/179] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:14:58 2022] GET /identity/v3/projects/4f42e0264e93481ea152c83b3ae53059 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:58.320801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ad89d-ca99-46fe-aaf2-5a1b387be55d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:58.424133 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ad89d-ca99-46fe-aaf2-5a1b387be55d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:58.461150 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dbf7d4-48bb-4739-85f6-05af8f20b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:58.461951 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbf7d4-48bb-4739-85f6-05af8f20b335 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:58.462129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbf7d4-48bb-4739-85f6-05af8f20b335 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:58.462340 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dbf7d4-48bb-4739-85f6-05af8f20b335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:58.463781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbf7d4-48bb-4739-85f6-05af8f20b335 None admin] RBAC: Adding query filter params (role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=4f42e0264e93481ea152c83b3ae53059, user.id=cf20a77d54c04b289feef5d1d0955a73) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:58.464096 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbf7d4-48bb-4739-85f6-05af8f20b335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:58.465652 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dbf7d4-48bb-4739-85f6-05af8f20b335 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:58.472073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 94/180] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:14:58 2022] GET /identity/v3/role_assignments?user.id=cf20a77d54c04b289feef5d1d0955a73&scope.project.id=4f42e0264e93481ea152c83b3ae53059&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:58.871253 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c531b-6a47-4817-945b-74630bddb2b5 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:58.871965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c531b-6a47-4817-945b-74630bddb2b5 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:58.872509 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c531b-6a47-4817-945b-74630bddb2b5 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:58.874278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 87/181] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:58.880605 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba5c2-481c-4521-aad4-d771f3315f8e None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:58.880957 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba5c2-481c-4521-aad4-d771f3315f8e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:58.881197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba5c2-481c-4521-aad4-d771f3315f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:58.907049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-646ba5c2-481c-4521-aad4-d771f3315f8e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:58.908247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-646ba5c2-481c-4521-aad4-d771f3315f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:58.988352 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 95/182] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:59.012035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c531b-6a47-4817-945b-74630bddb2b5 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.073316 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8b98-9926-4706-bfe6-822cfac57b5d None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.073853 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8b98-9926-4706-bfe6-822cfac57b5d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.074289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8b98-9926-4706-bfe6-822cfac57b5d None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.076024 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 96/183] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:59.081369 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce1fe1-3c23-428d-98e6-d696c1c21e23 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.081852 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce1fe1-3c23-428d-98e6-d696c1c21e23 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.082311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce1fe1-3c23-428d-98e6-d696c1c21e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.100343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-75ce1fe1-3c23-428d-98e6-d696c1c21e23 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:59.102462 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-75ce1fe1-3c23-428d-98e6-d696c1c21e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:59.133245 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c531b-6a47-4817-945b-74630bddb2b5 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.167774 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b9baf-73fa-4712-bb29-aef403da96eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.168884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b9baf-73fa-4712-bb29-aef403da96eb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.169347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b9baf-73fa-4712-bb29-aef403da96eb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.169805 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b9baf-73fa-4712-bb29-aef403da96eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.180349 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b9baf-73fa-4712-bb29-aef403da96eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:59.183522 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 97/184] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-8c1b9baf-73fa-4712-bb29-aef403da96eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:59.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:59.190544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:14:59.191663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:59.194407 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 88/185] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:59.205089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce1fe1-3c23-428d-98e6-d696c1c21e23 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.210279 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b9baf-73fa-4712-bb29-aef403da96eb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.312814 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b9baf-73fa-4712-bb29-aef403da96eb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.322989 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce1fe1-3c23-428d-98e6-d696c1c21e23 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.347020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f664e17-d3e2-4c8b-8c79-494ba35cd15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.347815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f664e17-d3e2-4c8b-8c79-494ba35cd15b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.348102 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f664e17-d3e2-4c8b-8c79-494ba35cd15b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.348378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f664e17-d3e2-4c8b-8c79-494ba35cd15b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.356860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc4a60-c6bf-4d5c-9bf4-adc90649c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.357664 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc4a60-c6bf-4d5c-9bf4-adc90649c53f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.357955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc4a60-c6bf-4d5c-9bf4-adc90649c53f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.358238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc4a60-c6bf-4d5c-9bf4-adc90649c53f None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.358964 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f664e17-d3e2-4c8b-8c79-494ba35cd15b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:59.359086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc4a60-c6bf-4d5c-9bf4-adc90649c53f None admin] RBAC: Adding query filter params (name=reader) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:59.359292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc4a60-c6bf-4d5c-9bf4-adc90649c53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:59.360684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc4a60-c6bf-4d5c-9bf4-adc90649c53f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-8f664e17-d3e2-4c8b-8c79-494ba35cd15b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:59.366667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:59.367925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 89/186] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:59.374228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec542e-ce8b-4958-94c2-a0240468ae73 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.375037 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec542e-ce8b-4958-94c2-a0240468ae73 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.375037 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec542e-ce8b-4958-94c2-a0240468ae73 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.375037 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 90/187] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:59.375037 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec542e-ce8b-4958-94c2-a0240468ae73 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.378160 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 98/188] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:59.379473 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49746db-a592-4685-8ec4-1052bfb5dc55 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.379831 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49746db-a592-4685-8ec4-1052bfb5dc55 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.380181 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49746db-a592-4685-8ec4-1052bfb5dc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.396297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-c49746db-a592-4685-8ec4-1052bfb5dc55 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:59.397442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-c49746db-a592-4685-8ec4-1052bfb5dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:59.468827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 99/189] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:59.470402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49746db-a592-4685-8ec4-1052bfb5dc55 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.489899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec542e-ce8b-4958-94c2-a0240468ae73 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.522776 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c52e-ed34-4dde-a920-897855ccac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.523442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c52e-ed34-4dde-a920-897855ccac68 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.523725 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c52e-ed34-4dde-a920-897855ccac68 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.524004 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c52e-ed34-4dde-a920-897855ccac68 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.524758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0c52e-ed34-4dde-a920-897855ccac68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:59.525092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0c52e-ed34-4dde-a920-897855ccac68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:59.526516 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0c52e-ed34-4dde-a920-897855ccac68 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:59.534269 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 91/190] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:14:59.535452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e423a-deff-422b-a77c-61764f0ccf73 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.535743 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e423a-deff-422b-a77c-61764f0ccf73 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.536024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e423a-deff-422b-a77c-61764f0ccf73 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.537317 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 92/191] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:14:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:14:59.538985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e423a-deff-422b-a77c-61764f0ccf73 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.566277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49746db-a592-4685-8ec4-1052bfb5dc55 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.600669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecfb14-68b2-4a57-a000-6a700b2f02ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.601568 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecfb14-68b2-4a57-a000-6a700b2f02ec None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.601850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecfb14-68b2-4a57-a000-6a700b2f02ec None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.602123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecfb14-68b2-4a57-a000-6a700b2f02ec None admin] PATH_INFO: `/v3/users/48709a2b22dd472fa3736d7bbf317557` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.603825 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecfb14-68b2-4a57-a000-6a700b2f02ec None admin] RBAC: Authorizing `identity:get_user(user_id=48709a2b22dd472fa3736d7bbf317557)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:59.605316 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecfb14-68b2-4a57-a000-6a700b2f02ec None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:59.606946 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 100/192] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/users/48709a2b22dd472fa3736d7bbf317557 => generated 289 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:59.611585 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecfb14-68b2-4a57-a000-6a700b2f02ec None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.642067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e423a-deff-422b-a77c-61764f0ccf73 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.673830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cde10b-5df8-4ff6-837a-23b9b4887fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.674661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cde10b-5df8-4ff6-837a-23b9b4887fb6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.674943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cde10b-5df8-4ff6-837a-23b9b4887fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.675225 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cde10b-5df8-4ff6-837a-23b9b4887fb6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.685581 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cde10b-5df8-4ff6-837a-23b9b4887fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b6cde10b-5df8-4ff6-837a-23b9b4887fb6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:14:59.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:14:59.694667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:14:59.695565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 93/193] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:14:59.696302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0da07-280a-4b41-ae08-b8cb0c019780 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.696302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0da07-280a-4b41-ae08-b8cb0c019780 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.696549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0da07-280a-4b41-ae08-b8cb0c019780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.704586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecfb14-68b2-4a57-a000-6a700b2f02ec None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.715599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-6bb0da07-280a-4b41-ae08-b8cb0c019780 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:14:59.716831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-6bb0da07-280a-4b41-ae08-b8cb0c019780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:14:59.739368 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b6667-46a5-4194-90c4-d1d9608bfae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.740243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6667-46a5-4194-90c4-d1d9608bfae0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.740523 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6667-46a5-4194-90c4-d1d9608bfae0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.740798 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6667-46a5-4194-90c4-d1d9608bfae0 None admin] PATH_INFO: `/v3/users/24fadea75260432a92706c933c9cbfa6` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.754589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b6667-46a5-4194-90c4-d1d9608bfae0 None admin] RBAC: Authorizing `identity:get_user(user_id=24fadea75260432a92706c933c9cbfa6)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:59.756040 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b6667-46a5-4194-90c4-d1d9608bfae0 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:59.757750 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 101/194] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/users/24fadea75260432a92706c933c9cbfa6 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:59.757750 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b6667-46a5-4194-90c4-d1d9608bfae0 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.782793 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 94/195] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:14:59.783650 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0da07-280a-4b41-ae08-b8cb0c019780 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.860780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b6667-46a5-4194-90c4-d1d9608bfae0 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.876568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0da07-280a-4b41-ae08-b8cb0c019780 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:14:59.892892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a593d01-09a8-4aff-8131-81c3b971af2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.893562 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a593d01-09a8-4aff-8131-81c3b971af2b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.893848 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a593d01-09a8-4aff-8131-81c3b971af2b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.894117 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a593d01-09a8-4aff-8131-81c3b971af2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.908558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:14:59.909247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:14:59.909531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:14:59.909804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:14:59.910553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:14:59.910856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:59.912252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:14:59.920597 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 95/196] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:14:59.920597 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:14:59.963484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a593d01-09a8-4aff-8131-81c3b971af2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:14:59.964880 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a593d01-09a8-4aff-8131-81c3b971af2b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.013387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176d3ae-1021-4d69-9b80-1d4dcbaffd48 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.045031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139da97-836b-48d5-897a-73c5a5a4c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.045939 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139da97-836b-48d5-897a-73c5a5a4c50d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.046355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139da97-836b-48d5-897a-73c5a5a4c50d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.046757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139da97-836b-48d5-897a-73c5a5a4c50d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.057072 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 102/197] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:00.057844 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a593d01-09a8-4aff-8131-81c3b971af2b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:00.127192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139da97-836b-48d5-897a-73c5a5a4c50d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.128672 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139da97-836b-48d5-897a-73c5a5a4c50d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.167445 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a593d01-09a8-4aff-8131-81c3b971af2b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.199555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00245540-5f16-40a1-9f74-09e8f9dda79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.200337 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00245540-5f16-40a1-9f74-09e8f9dda79d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.200616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00245540-5f16-40a1-9f74-09e8f9dda79d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.200888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00245540-5f16-40a1-9f74-09e8f9dda79d None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.202084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00245540-5f16-40a1-9f74-09e8f9dda79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.203538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00245540-5f16-40a1-9f74-09e8f9dda79d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.205159 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 103/198] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:00.205923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00245540-5f16-40a1-9f74-09e8f9dda79d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:00.230522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 96/199] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:00.230522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139da97-836b-48d5-897a-73c5a5a4c50d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:00.300279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00245540-5f16-40a1-9f74-09e8f9dda79d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.321023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139da97-836b-48d5-897a-73c5a5a4c50d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.331884 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f04358-2f43-4a77-8f3e-758fb0190db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.332742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f04358-2f43-4a77-8f3e-758fb0190db9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.333020 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f04358-2f43-4a77-8f3e-758fb0190db9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.333323 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f04358-2f43-4a77-8f3e-758fb0190db9 None admin] PATH_INFO: `/v3/users/24fadea75260432a92706c933c9cbfa6` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.334515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f04358-2f43-4a77-8f3e-758fb0190db9 None admin] RBAC: Authorizing `identity:get_user(user_id=24fadea75260432a92706c933c9cbfa6)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.335943 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f04358-2f43-4a77-8f3e-758fb0190db9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.337593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 104/200] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:00 2022] GET /identity/v3/users/24fadea75260432a92706c933c9cbfa6 => generated 283 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:00.338896 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f04358-2f43-4a77-8f3e-758fb0190db9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:00.354667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b1c87-fd6c-4f78-842b-32986142fdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.355529 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b1c87-fd6c-4f78-842b-32986142fdee None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.355819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b1c87-fd6c-4f78-842b-32986142fdee None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.356095 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b1c87-fd6c-4f78-842b-32986142fdee None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.357331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b1c87-fd6c-4f78-842b-32986142fdee None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.358767 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b1c87-fd6c-4f78-842b-32986142fdee None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.360456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 97/201] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:00 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:00.361963 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b1c87-fd6c-4f78-842b-32986142fdee None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:00.431950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f04358-2f43-4a77-8f3e-758fb0190db9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.456442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b1c87-fd6c-4f78-842b-32986142fdee None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.471232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3852dc-ced1-4f0d-989a-4a66c228a56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.471898 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3852dc-ced1-4f0d-989a-4a66c228a56b None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.472175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3852dc-ced1-4f0d-989a-4a66c228a56b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.472443 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3852dc-ced1-4f0d-989a-4a66c228a56b None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.473191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3852dc-ced1-4f0d-989a-4a66c228a56b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.474622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3852dc-ced1-4f0d-989a-4a66c228a56b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.490362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6eb19-101b-412a-ba0b-9b97f1916020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.491384 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6eb19-101b-412a-ba0b-9b97f1916020 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.491667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6eb19-101b-412a-ba0b-9b97f1916020 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.491944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6eb19-101b-412a-ba0b-9b97f1916020 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.493171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6eb19-101b-412a-ba0b-9b97f1916020 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.494874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6eb19-101b-412a-ba0b-9b97f1916020 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.496558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 98/202] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:00 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:00.497428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6eb19-101b-412a-ba0b-9b97f1916020 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:00.517075 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 105/203] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:00 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:15:00.517075 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3852dc-ced1-4f0d-989a-4a66c228a56b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:00.606628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6eb19-101b-412a-ba0b-9b97f1916020 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.618694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3852dc-ced1-4f0d-989a-4a66c228a56b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.643007 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0281bb3-46ad-4d8d-bd88-cd7af8635bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.643807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0281bb3-46ad-4d8d-bd88-cd7af8635bbe None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.644089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0281bb3-46ad-4d8d-bd88-cd7af8635bbe None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.644366 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0281bb3-46ad-4d8d-bd88-cd7af8635bbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.655014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.655683 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.655960 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.656233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.657097 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] RBAC: Adding query filter params (role.id=f27561b987b14525a832c7d37ff8c62c, scope.project.id=84caf63eb1634db7822fe78409be9c77, user.id=48709a2b22dd472fa3736d7bbf317557) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:00.657402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.658883 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.675099 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 106/204] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:00 2022] GET /identity/v3/role_assignments?user.id=48709a2b22dd472fa3736d7bbf317557&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=f27561b987b14525a832c7d37ff8c62c => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:00.675099 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:00.743373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0281bb3-46ad-4d8d-bd88-cd7af8635bbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.745812 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0281bb3-46ad-4d8d-bd88-cd7af8635bbe None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.791656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b271a-171b-435b-826c-b459e4b8cf70 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:00.828549 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:00.829338 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:00.829618 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:00.829889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:00.830744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=66c8b57a8c8e424baf3afade22505814, user.id=24fadea75260432a92706c933c9cbfa6) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:00.831052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:00.832536 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:00.838929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 107/205] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:00 2022] GET /identity/v3/role_assignments?user.id=24fadea75260432a92706c933c9cbfa6&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:00.887173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 99/206] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:00.893095 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:01.010902 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404affb-7b15-41d9-b10b-ad7df70967a5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:01.050397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200feb51-b332-4030-b89c-3ccdc289a351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:01.051628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200feb51-b332-4030-b89c-3ccdc289a351 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:01.052123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200feb51-b332-4030-b89c-3ccdc289a351 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:01.052640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200feb51-b332-4030-b89c-3ccdc289a351 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:01.054123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200feb51-b332-4030-b89c-3ccdc289a351 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:01.055847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200feb51-b332-4030-b89c-3ccdc289a351 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:01.058719 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 108/207] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:00 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:01.064194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0281bb3-46ad-4d8d-bd88-cd7af8635bbe None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:01.172066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0281bb3-46ad-4d8d-bd88-cd7af8635bbe None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:01.208284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d5dc8-a9e5-4b75-80aa-6c498cc5274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:01.209361 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5dc8-a9e5-4b75-80aa-6c498cc5274c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:01.209863 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5dc8-a9e5-4b75-80aa-6c498cc5274c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:01.210369 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5dc8-a9e5-4b75-80aa-6c498cc5274c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:01.211448 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d5dc8-a9e5-4b75-80aa-6c498cc5274c None admin] RBAC: Adding query filter params (role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=66c8b57a8c8e424baf3afade22505814, user.id=24fadea75260432a92706c933c9cbfa6) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:01.211986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d5dc8-a9e5-4b75-80aa-6c498cc5274c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:01.213738 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d5dc8-a9e5-4b75-80aa-6c498cc5274c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:01.220050 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 100/208] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:01 2022] GET /identity/v3/role_assignments?user.id=24fadea75260432a92706c933c9cbfa6&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=c8ee1507247e4b34a213a8033c46378f => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:02.971953 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6500c-d4de-43e8-869c-8aa7f40b0db1 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:02.972500 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6500c-d4de-43e8-869c-8aa7f40b0db1 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:02.972500 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6500c-d4de-43e8-869c-8aa7f40b0db1 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:02.974319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 109/209] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:02.981073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4363ec-96b8-42f3-ba3a-8f736c248d81 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:02.981635 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4363ec-96b8-42f3-ba3a-8f736c248d81 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:02.981635 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4363ec-96b8-42f3-ba3a-8f736c248d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:02.999505 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-5b4363ec-96b8-42f3-ba3a-8f736c248d81 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:03.000693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-5b4363ec-96b8-42f3-ba3a-8f736c248d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:03.049455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a0d4-9e04-4c04-ab07-4876f44cff43 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.049811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a0d4-9e04-4c04-ab07-4876f44cff43 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.050017 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a0d4-9e04-4c04-ab07-4876f44cff43 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.051410 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 110/210] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:03.056534 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d037-ffc9-4f49-bcf3-d761824fcfdb None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.056822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d037-ffc9-4f49-bcf3-d761824fcfdb None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.057165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d037-ffc9-4f49-bcf3-d761824fcfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.073173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 101/211] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:03.076224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-c4b3d037-ffc9-4f49-bcf3-d761824fcfdb None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:03.077475 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-c4b3d037-ffc9-4f49-bcf3-d761824fcfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:03.096507 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4363ec-96b8-42f3-ba3a-8f736c248d81 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.156174 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 111/212] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:03.157714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ce2e6-13ef-40d0-a29d-45da9c6d34d5 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.158001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ce2e6-13ef-40d0-a29d-45da9c6d34d5 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.158274 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ce2e6-13ef-40d0-a29d-45da9c6d34d5 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.159680 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 112/213] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:03.164850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94254e2c-cf19-4fe4-90c0-ef200700a924 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.165169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94254e2c-cf19-4fe4-90c0-ef200700a924 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.165449 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94254e2c-cf19-4fe4-90c0-ef200700a924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.182600 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-94254e2c-cf19-4fe4-90c0-ef200700a924 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:03.183794 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-94254e2c-cf19-4fe4-90c0-ef200700a924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:03.201336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4363ec-96b8-42f3-ba3a-8f736c248d81 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.236149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb307f0-9d8e-431b-8df3-77d9a0ad5714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.236859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb307f0-9d8e-431b-8df3-77d9a0ad5714 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.237179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb307f0-9d8e-431b-8df3-77d9a0ad5714 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.237455 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb307f0-9d8e-431b-8df3-77d9a0ad5714 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.248358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 113/214] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:03.249025 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94254e2c-cf19-4fe4-90c0-ef200700a924 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.257417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb307f0-9d8e-431b-8df3-77d9a0ad5714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-4eb307f0-9d8e-431b-8df3-77d9a0ad5714 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:03.268085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:03.269254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:03.270207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:03.270943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 102/215] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:03.271790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb307f0-9d8e-431b-8df3-77d9a0ad5714 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.356795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94254e2c-cf19-4fe4-90c0-ef200700a924 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.376591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb307f0-9d8e-431b-8df3-77d9a0ad5714 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.394434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1023-268a-4672-88a2-c3652dcb5e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.395233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1023-268a-4672-88a2-c3652dcb5e64 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.395516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1023-268a-4672-88a2-c3652dcb5e64 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.395790 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1023-268a-4672-88a2-c3652dcb5e64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.397684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1023-268a-4672-88a2-c3652dcb5e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:03.399648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1023-268a-4672-88a2-c3652dcb5e64 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:03.403518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 114/216] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:03.404417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1023-268a-4672-88a2-c3652dcb5e64 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.415573 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1cac-2bb5-4487-a008-9639ea3f0ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.416676 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1cac-2bb5-4487-a008-9639ea3f0ec3 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.417239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1cac-2bb5-4487-a008-9639ea3f0ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.417705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1cac-2bb5-4487-a008-9639ea3f0ec3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.429686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d1cac-2bb5-4487-a008-9639ea3f0ec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d08d1cac-2bb5-4487-a008-9639ea3f0ec3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:03.437970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:03.438970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:03.439871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:03.443600 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 103/217] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:03.444505 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1cac-2bb5-4487-a008-9639ea3f0ec3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.499863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1023-268a-4672-88a2-c3652dcb5e64 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.532052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.532717 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.532996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.533298 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.534045 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:03.534347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:03.535705 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:03.539477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1cac-2bb5-4487-a008-9639ea3f0ec3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.543364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 115/218] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:03.544149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.571411 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3779c-a854-4e89-924f-73ac867126ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.572610 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3779c-a854-4e89-924f-73ac867126ac None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.572894 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3779c-a854-4e89-924f-73ac867126ac None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.573186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3779c-a854-4e89-924f-73ac867126ac None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.573929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3779c-a854-4e89-924f-73ac867126ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:03.575347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3779c-a854-4e89-924f-73ac867126ac None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:03.596096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 104/219] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:03 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:15:03.597316 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5399-9b91-4051-bbbd-aea71d639a37 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.597612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5399-9b91-4051-bbbd-aea71d639a37 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.597891 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5399-9b91-4051-bbbd-aea71d639a37 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.599152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 105/220] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:03.601442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5399-9b91-4051-bbbd-aea71d639a37 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.647312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffd2f3-1f5d-4fbd-aece-f585892314c1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.683832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b1e41-21de-484c-a750-2a0030882bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.684527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b1e41-21de-484c-a750-2a0030882bc5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.684817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b1e41-21de-484c-a750-2a0030882bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.685127 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b1e41-21de-484c-a750-2a0030882bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.685921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b1e41-21de-484c-a750-2a0030882bc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:03.686221 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b1e41-21de-484c-a750-2a0030882bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:03.687704 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b1e41-21de-484c-a750-2a0030882bc5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:03.701850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 116/221] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:03.701850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90b8d-235a-48d9-b77a-df9fab31212e None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.701850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90b8d-235a-48d9-b77a-df9fab31212e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.701850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90b8d-235a-48d9-b77a-df9fab31212e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.715865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-4fc90b8d-235a-48d9-b77a-df9fab31212e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:03.716268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5399-9b91-4051-bbbd-aea71d639a37 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.717117 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-4fc90b8d-235a-48d9-b77a-df9fab31212e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:03.756854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a1c92-a667-4fb3-a4e4-64742b1929e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.757686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a1c92-a667-4fb3-a4e4-64742b1929e8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.757974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a1c92-a667-4fb3-a4e4-64742b1929e8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.758251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a1c92-a667-4fb3-a4e4-64742b1929e8 None admin] PATH_INFO: `/v3/users/4e2a0d46e4e541b0b147639282d73c31` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.776817 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a1c92-a667-4fb3-a4e4-64742b1929e8 None admin] RBAC: Authorizing `identity:get_user(user_id=4e2a0d46e4e541b0b147639282d73c31)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:03.778437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a1c92-a667-4fb3-a4e4-64742b1929e8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:03.780320 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 106/222] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/users/4e2a0d46e4e541b0b147639282d73c31 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:03.780952 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a1c92-a667-4fb3-a4e4-64742b1929e8 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.796044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 117/223] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:03.796858 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc90b8d-235a-48d9-b77a-df9fab31212e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:03.903059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a1c92-a667-4fb3-a4e4-64742b1929e8 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.906033 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc90b8d-235a-48d9-b77a-df9fab31212e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:03.947402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59674e-ade4-4198-9aa7-980b0b9a0d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.948290 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59674e-ade4-4198-9aa7-980b0b9a0d98 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.948571 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59674e-ade4-4198-9aa7-980b0b9a0d98 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.948845 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59674e-ade4-4198-9aa7-980b0b9a0d98 None admin] PATH_INFO: `/v3/users/24fadea75260432a92706c933c9cbfa6` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.950310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59674e-ade4-4198-9aa7-980b0b9a0d98 None admin] RBAC: Authorizing `identity:get_user(user_id=24fadea75260432a92706c933c9cbfa6)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:03.951571 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59674e-ade4-4198-9aa7-980b0b9a0d98 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:03.951698 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5b61-cc1b-4e17-a940-03815b892bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:03.952330 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5b61-cc1b-4e17-a940-03815b892bc3 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:03.952611 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5b61-cc1b-4e17-a940-03815b892bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:03.952889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5b61-cc1b-4e17-a940-03815b892bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:03.953443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 107/224] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/users/24fadea75260432a92706c933c9cbfa6 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:03.954045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59674e-ade4-4198-9aa7-980b0b9a0d98 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.025001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5b61-cc1b-4e17-a940-03815b892bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:04.027058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5b61-cc1b-4e17-a940-03815b892bc3 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:04.062373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59674e-ade4-4198-9aa7-980b0b9a0d98 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.099706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3633e-a1a4-45e7-8362-4c350f25a595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.100933 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3633e-a1a4-45e7-8362-4c350f25a595 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.101447 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3633e-a1a4-45e7-8362-4c350f25a595 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.101994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3633e-a1a4-45e7-8362-4c350f25a595 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.113288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3633e-a1a4-45e7-8362-4c350f25a595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-cab3633e-a1a4-45e7-8362-4c350f25a595 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:04.123061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:04.124119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:04.125028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:04.127865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 108/225] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:04.128935 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3633e-a1a4-45e7-8362-4c350f25a595 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.141342 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 118/226] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:04.142361 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5b61-cc1b-4e17-a940-03815b892bc3 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.231166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3633e-a1a4-45e7-8362-4c350f25a595 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.246518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5b61-cc1b-4e17-a940-03815b892bc3 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.272605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373805-7158-4e63-aeef-4ba2682c27eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.273538 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373805-7158-4e63-aeef-4ba2682c27eb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.273815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373805-7158-4e63-aeef-4ba2682c27eb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.274091 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373805-7158-4e63-aeef-4ba2682c27eb None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.275556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373805-7158-4e63-aeef-4ba2682c27eb None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:04.276969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373805-7158-4e63-aeef-4ba2682c27eb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:04.283682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 109/227] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:04 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:04.284665 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373805-7158-4e63-aeef-4ba2682c27eb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.286157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.287072 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.287566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.288068 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.289063 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:04.289600 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:04.291779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:04.304201 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 119/228] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:04.305381 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.408423 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948c0d3-8eee-4112-bb56-cbd76b11557c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.417318 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373805-7158-4e63-aeef-4ba2682c27eb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.443967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d83bb8-eab2-483e-97cd-dd1fb35d2307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.445296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d83bb8-eab2-483e-97cd-dd1fb35d2307 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.445892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d83bb8-eab2-483e-97cd-dd1fb35d2307 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.446383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d83bb8-eab2-483e-97cd-dd1fb35d2307 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814/users/24fadea75260432a92706c933c9cbfa6/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.449235 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d83bb8-eab2-483e-97cd-dd1fb35d2307 None admin] RBAC: Authorizing `identity:create_grant(project_id=66c8b57a8c8e424baf3afade22505814, user_id=24fadea75260432a92706c933c9cbfa6, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:04.450911 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d83bb8-eab2-483e-97cd-dd1fb35d2307 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:04.458212 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f99cf-70c9-4abd-9f50-5b52f9372c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.459140 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f99cf-70c9-4abd-9f50-5b52f9372c76 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.459411 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f99cf-70c9-4abd-9f50-5b52f9372c76 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.459685 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f99cf-70c9-4abd-9f50-5b52f9372c76 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.461144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f99cf-70c9-4abd-9f50-5b52f9372c76 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:04.462780 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f99cf-70c9-4abd-9f50-5b52f9372c76 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:04.469098 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 110/229] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:04 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:04.469098 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f99cf-70c9-4abd-9f50-5b52f9372c76 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.473629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 120/230] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:04 2022] PUT /identity/v3/projects/66c8b57a8c8e424baf3afade22505814/users/24fadea75260432a92706c933c9cbfa6/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:04.474289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d83bb8-eab2-483e-97cd-dd1fb35d2307 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.596095 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f99cf-70c9-4abd-9f50-5b52f9372c76 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.638533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28db0-938d-4fab-b221-b3a938db2486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.639923 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28db0-938d-4fab-b221-b3a938db2486 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.640444 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28db0-938d-4fab-b221-b3a938db2486 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.640983 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28db0-938d-4fab-b221-b3a938db2486 None admin] PATH_INFO: `/v3/users/24fadea75260432a92706c933c9cbfa6` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.642904 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da28db0-938d-4fab-b221-b3a938db2486 None admin] RBAC: Authorizing `identity:get_user(user_id=24fadea75260432a92706c933c9cbfa6)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:04.644642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da28db0-938d-4fab-b221-b3a938db2486 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:04.647387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 111/231] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:04 2022] GET /identity/v3/users/24fadea75260432a92706c933c9cbfa6 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:04.652966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28db0-938d-4fab-b221-b3a938db2486 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.747591 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d83bb8-eab2-483e-97cd-dd1fb35d2307 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.770925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28db0-938d-4fab-b221-b3a938db2486 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.783898 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.784559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.784839 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.785142 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.785979 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] RBAC: Adding query filter params (role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=66c8b57a8c8e424baf3afade22505814, user.id=4e2a0d46e4e541b0b147639282d73c31) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:04.786291 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:04.787738 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:04.793767 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 121/232] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:04 2022] GET /identity/v3/role_assignments?user.id=4e2a0d46e4e541b0b147639282d73c31&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:04.820069 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f57d5c-15b6-41ec-bd98-6971880d510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.820979 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57d5c-15b6-41ec-bd98-6971880d510a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.821317 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57d5c-15b6-41ec-bd98-6971880d510a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.821599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57d5c-15b6-41ec-bd98-6971880d510a None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.823486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f57d5c-15b6-41ec-bd98-6971880d510a None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:04.825025 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f57d5c-15b6-41ec-bd98-6971880d510a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:04.827191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 112/233] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:04 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:04.833067 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:04.947150 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975a782-064e-4c55-8073-743a7fc7b62b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:04.992463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37df2d-b611-4bfc-89b0-bc85ca6b7b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:04.993414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37df2d-b611-4bfc-89b0-bc85ca6b7b25 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:04.993696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37df2d-b611-4bfc-89b0-bc85ca6b7b25 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:04.994898 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37df2d-b611-4bfc-89b0-bc85ca6b7b25 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814/users/24fadea75260432a92706c933c9cbfa6/roles/c8ee1507247e4b34a213a8033c46378f` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:04.999040 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37df2d-b611-4bfc-89b0-bc85ca6b7b25 None admin] RBAC: Authorizing `identity:create_grant(project_id=66c8b57a8c8e424baf3afade22505814, user_id=24fadea75260432a92706c933c9cbfa6, role_id=c8ee1507247e4b34a213a8033c46378f)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:05.000536 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37df2d-b611-4bfc-89b0-bc85ca6b7b25 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:05.026196 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 122/234] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:04 2022] PUT /identity/v3/projects/66c8b57a8c8e424baf3afade22505814/users/24fadea75260432a92706c933c9cbfa6/roles/c8ee1507247e4b34a213a8033c46378f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:06.138555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc358d98-0284-4b45-a37d-19de7fb6360b None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:06.139112 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc358d98-0284-4b45-a37d-19de7fb6360b None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:06.139112 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc358d98-0284-4b45-a37d-19de7fb6360b None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:06.140775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 113/235] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:06.146593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4fbc9-caf2-45ca-a80a-71723e8065fa None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:06.147529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4fbc9-caf2-45ca-a80a-71723e8065fa None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:06.148024 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4fbc9-caf2-45ca-a80a-71723e8065fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:06.172074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-38a4fbc9-caf2-45ca-a80a-71723e8065fa None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:06.173678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-38a4fbc9-caf2-45ca-a80a-71723e8065fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:06.271270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 123/236] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:06.317751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc358d98-0284-4b45-a37d-19de7fb6360b None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:06.357382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6112-fc89-4e60-ad9c-4c09dde14cf6 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:06.357731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6112-fc89-4e60-ad9c-4c09dde14cf6 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:06.357946 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6112-fc89-4e60-ad9c-4c09dde14cf6 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:06.359598 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 124/237] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:06.364968 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f60caf-4d3a-4cd0-817b-f07aab2deca6 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:06.365290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f60caf-4d3a-4cd0-817b-f07aab2deca6 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:06.365574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f60caf-4d3a-4cd0-817b-f07aab2deca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:06.402030 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-e4f60caf-4d3a-4cd0-817b-f07aab2deca6 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:06.403242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-e4f60caf-4d3a-4cd0-817b-f07aab2deca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:06.435547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc358d98-0284-4b45-a37d-19de7fb6360b None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:06.470453 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7701-9e9b-48a8-948a-095fb8de47eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:06.471624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7701-9e9b-48a8-948a-095fb8de47eb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:06.472137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7701-9e9b-48a8-948a-095fb8de47eb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:06.472620 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7701-9e9b-48a8-948a-095fb8de47eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:06.483106 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a7701-9e9b-48a8-948a-095fb8de47eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-875a7701-9e9b-48a8-948a-095fb8de47eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:06.491048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:06.492048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:06.492951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:06.496026 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 114/238] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:06.502167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7701-9e9b-48a8-948a-095fb8de47eb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:06.530881 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 125/239] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:06.578484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f60caf-4d3a-4cd0-817b-f07aab2deca6 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:06.615468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7701-9e9b-48a8-948a-095fb8de47eb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:06.664861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:06.665956 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:06.666489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:06.666979 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:06.667951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] RBAC: Adding query filter params (name=reader) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:06.668500 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:06.670261 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:06.679443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 115/240] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:06.686140 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:06.790049 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c1264-4eb8-457d-a0a2-6723f5e30eda None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:06.793308 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f60caf-4d3a-4cd0-817b-f07aab2deca6 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:06.825751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c308bbe-7ed4-4be8-badb-b2adcdba6f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:06.826995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c308bbe-7ed4-4be8-badb-b2adcdba6f8b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:06.827486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c308bbe-7ed4-4be8-badb-b2adcdba6f8b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:06.827966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c308bbe-7ed4-4be8-badb-b2adcdba6f8b None admin] PATH_INFO: `/v3/users/de2bf9de561841cebb16127e042d91f4` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:06.842059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c308bbe-7ed4-4be8-badb-b2adcdba6f8b None admin] RBAC: Authorizing `identity:get_user(user_id=de2bf9de561841cebb16127e042d91f4)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:06.843893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c308bbe-7ed4-4be8-badb-b2adcdba6f8b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:06.846182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 116/241] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:06 2022] GET /identity/v3/users/de2bf9de561841cebb16127e042d91f4 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:06.851254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c308bbe-7ed4-4be8-badb-b2adcdba6f8b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:06.863310 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e390a3-4100-4ffb-9688-2a4198623429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:06.864477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e390a3-4100-4ffb-9688-2a4198623429 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:06.864971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e390a3-4100-4ffb-9688-2a4198623429 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:06.865487 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e390a3-4100-4ffb-9688-2a4198623429 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:06.869708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e390a3-4100-4ffb-9688-2a4198623429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:06.871416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e390a3-4100-4ffb-9688-2a4198623429 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:06.877350 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 126/242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:06.880183 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e390a3-4100-4ffb-9688-2a4198623429 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:06.967554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c308bbe-7ed4-4be8-badb-b2adcdba6f8b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:07.002310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f48825-08b5-45d3-b88a-5abb11f46453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:07.003349 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f48825-08b5-45d3-b88a-5abb11f46453 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.003863 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f48825-08b5-45d3-b88a-5abb11f46453 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.004363 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f48825-08b5-45d3-b88a-5abb11f46453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.007115 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e390a3-4100-4ffb-9688-2a4198623429 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:07.048764 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4725fb-ea4c-49bd-b9c4-e7b2c9a6d81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:07.049391 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4725fb-ea4c-49bd-b9c4-e7b2c9a6d81c None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.049651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4725fb-ea4c-49bd-b9c4-e7b2c9a6d81c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.049928 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4725fb-ea4c-49bd-b9c4-e7b2c9a6d81c None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.050721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4725fb-ea4c-49bd-b9c4-e7b2c9a6d81c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:07.052225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4725fb-ea4c-49bd-b9c4-e7b2c9a6d81c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:07.075794 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f48825-08b5-45d3-b88a-5abb11f46453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:07.078008 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f48825-08b5-45d3-b88a-5abb11f46453 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:07.091187 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 127/243] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:06 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:15:07.185310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 117/244] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:07.194508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4725fb-ea4c-49bd-b9c4-e7b2c9a6d81c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:07.313203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4725fb-ea4c-49bd-b9c4-e7b2c9a6d81c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:07.354846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c15d0-8751-4054-a460-4ba606ca3765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:07.355741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c15d0-8751-4054-a460-4ba606ca3765 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.356029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c15d0-8751-4054-a460-4ba606ca3765 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.356297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c15d0-8751-4054-a460-4ba606ca3765 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.357578 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c15d0-8751-4054-a460-4ba606ca3765 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:07.359071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c15d0-8751-4054-a460-4ba606ca3765 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:07.361134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 128/245] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:07 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:07.367117 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f48825-08b5-45d3-b88a-5abb11f46453 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:07.409475 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4de49c-af0f-49b9-8ec4-347ce353cfe6 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.409828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4de49c-af0f-49b9-8ec4-347ce353cfe6 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.410043 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4de49c-af0f-49b9-8ec4-347ce353cfe6 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.411439 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 129/246] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:07.417259 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e0bb-37d4-4ef1-bfbd-92a7a703d66b None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.417554 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e0bb-37d4-4ef1-bfbd-92a7a703d66b None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e0bb-37d4-4ef1-bfbd-92a7a703d66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.438637 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-8b31e0bb-37d4-4ef1-bfbd-92a7a703d66b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:07.439846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-8b31e0bb-37d4-4ef1-bfbd-92a7a703d66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:07.479818 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f48825-08b5-45d3-b88a-5abb11f46453 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:07.512256 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 130/247] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:07.515533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:07.516612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.517216 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.517739 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.518827 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] RBAC: Adding query filter params (role.id=f27561b987b14525a832c7d37ff8c62c, scope.project.id=66c8b57a8c8e424baf3afade22505814, user.id=de2bf9de561841cebb16127e042d91f4) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:07.519379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:07.521229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:07.528741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 118/248] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:07 2022] GET /identity/v3/role_assignments?user.id=de2bf9de561841cebb16127e042d91f4&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=f27561b987b14525a832c7d37ff8c62c => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:07.538036 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e0bb-37d4-4ef1-bfbd-92a7a703d66b None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:07.651624 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e0bb-37d4-4ef1-bfbd-92a7a703d66b None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:07.688556 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b667a-5371-45b5-9d99-0e7f534c7db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:07.689414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b667a-5371-45b5-9d99-0e7f534c7db2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.689702 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b667a-5371-45b5-9d99-0e7f534c7db2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.689979 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b667a-5371-45b5-9d99-0e7f534c7db2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.701621 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b667a-5371-45b5-9d99-0e7f534c7db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-103b667a-5371-45b5-9d99-0e7f534c7db2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:07.709515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:07.710569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:07.711477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 131/249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:07.717316 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:07.726363 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5fde-1039-4199-a7bd-0cad4b2a95d3 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.726714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5fde-1039-4199-a7bd-0cad4b2a95d3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.726918 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5fde-1039-4199-a7bd-0cad4b2a95d3 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.729192 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 132/250] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:07.734733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1830a4c-237b-44a8-b33e-3ad0e69e58ee None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.735015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1830a4c-237b-44a8-b33e-3ad0e69e58ee None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.735270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1830a4c-237b-44a8-b33e-3ad0e69e58ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.754777 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-e1830a4c-237b-44a8-b33e-3ad0e69e58ee None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:07.756489 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-e1830a4c-237b-44a8-b33e-3ad0e69e58ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:07.848433 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 133/251] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:07.856729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf7d46-7382-4815-a346-a9872e5bf01e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:07.871437 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1830a4c-237b-44a8-b33e-3ad0e69e58ee None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:07.905459 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:07.906840 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:07.907343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:07.907979 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:07.909110 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:07.909590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:07.911210 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:07.923453 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 119/252] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:07.929100 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:07.990860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1830a4c-237b-44a8-b33e-3ad0e69e58ee None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:08.037229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65735-e3ef-402b-ba34-f7f59ee29815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:08.038649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65735-e3ef-402b-ba34-f7f59ee29815 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.039145 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65735-e3ef-402b-ba34-f7f59ee29815 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.039617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65735-e3ef-402b-ba34-f7f59ee29815 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.048041 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7be05-b797-4eae-9e63-ad6219f8c64a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:08.051948 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b65735-e3ef-402b-ba34-f7f59ee29815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d3b65735-e3ef-402b-ba34-f7f59ee29815 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:08.060047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:08.061124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:08.062001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:08.064900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 134/253] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:08.070890 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65735-e3ef-402b-ba34-f7f59ee29815 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:08.085115 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acef96c-c9fb-432a-bdb5-54937c0c4a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:08.085965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef96c-c9fb-432a-bdb5-54937c0c4a36 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.086303 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef96c-c9fb-432a-bdb5-54937c0c4a36 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.086585 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef96c-c9fb-432a-bdb5-54937c0c4a36 None admin] PATH_INFO: `/v3/users/24fadea75260432a92706c933c9cbfa6` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.089076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acef96c-c9fb-432a-bdb5-54937c0c4a36 None admin] RBAC: Authorizing `identity:get_user(user_id=24fadea75260432a92706c933c9cbfa6)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:08.090577 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acef96c-c9fb-432a-bdb5-54937c0c4a36 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:08.093480 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 120/254] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:07 2022] GET /identity/v3/users/24fadea75260432a92706c933c9cbfa6 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:08.098282 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acef96c-c9fb-432a-bdb5-54937c0c4a36 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:08.194545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65735-e3ef-402b-ba34-f7f59ee29815 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:08.216129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acef96c-c9fb-432a-bdb5-54937c0c4a36 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:08.229207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f080a1-eb89-4759-9584-c27a76b77ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:08.229878 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f080a1-eb89-4759-9584-c27a76b77ff0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.230157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f080a1-eb89-4759-9584-c27a76b77ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f080a1-eb89-4759-9584-c27a76b77ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.231215 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f080a1-eb89-4759-9584-c27a76b77ff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:08.231479 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f080a1-eb89-4759-9584-c27a76b77ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:08.232868 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f080a1-eb89-4759-9584-c27a76b77ff0 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:08.241627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 135/255] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:08.242923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50819b1-0f5a-4277-9717-adf7c5c9db76 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.243173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50819b1-0f5a-4277-9717-adf7c5c9db76 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.243449 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50819b1-0f5a-4277-9717-adf7c5c9db76 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.245155 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 136/256] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:08.250696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba891d05-82e8-4f44-962e-3afecdd12b59 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.250948 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba891d05-82e8-4f44-962e-3afecdd12b59 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.251199 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba891d05-82e8-4f44-962e-3afecdd12b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.268594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-ba891d05-82e8-4f44-962e-3afecdd12b59 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:08.269919 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-ba891d05-82e8-4f44-962e-3afecdd12b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:08.270378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e923-8385-454a-97d4-f291d2644237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:08.271052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e923-8385-454a-97d4-f291d2644237 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.271337 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e923-8385-454a-97d4-f291d2644237 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.271617 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e923-8385-454a-97d4-f291d2644237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.341631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 137/257] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:08.342896 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba891d05-82e8-4f44-962e-3afecdd12b59 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:08.442094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5e923-8385-454a-97d4-f291d2644237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:08.443578 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5e923-8385-454a-97d4-f291d2644237 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:08.451932 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba891d05-82e8-4f44-962e-3afecdd12b59 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:08.493555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11896b04-3245-4dc0-9274-80fdb8153b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:08.495494 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11896b04-3245-4dc0-9274-80fdb8153b5b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.495774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11896b04-3245-4dc0-9274-80fdb8153b5b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.496226 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11896b04-3245-4dc0-9274-80fdb8153b5b None admin] PATH_INFO: `/v3/users/4e2a0d46e4e541b0b147639282d73c31` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.497687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11896b04-3245-4dc0-9274-80fdb8153b5b None admin] RBAC: Authorizing `identity:get_user(user_id=4e2a0d46e4e541b0b147639282d73c31)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:08.499139 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11896b04-3245-4dc0-9274-80fdb8153b5b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:08.501696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 138/258] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:08 2022] GET /identity/v3/users/4e2a0d46e4e541b0b147639282d73c31 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:08.502444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11896b04-3245-4dc0-9274-80fdb8153b5b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:08.582846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 121/259] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:15:08.583823 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e923-8385-454a-97d4-f291d2644237 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:08.623874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11896b04-3245-4dc0-9274-80fdb8153b5b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:08.666314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8ad15-d7aa-40de-8cb0-159bab21cd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:08.667488 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8ad15-d7aa-40de-8cb0-159bab21cd8a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.667770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8ad15-d7aa-40de-8cb0-159bab21cd8a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.668313 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8ad15-d7aa-40de-8cb0-159bab21cd8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.679692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8ad15-d7aa-40de-8cb0-159bab21cd8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d0a8ad15-d7aa-40de-8cb0-159bab21cd8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:08.689178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:08.691515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 139/260] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:08.693699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8ad15-d7aa-40de-8cb0-159bab21cd8a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:08.715131 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e923-8385-454a-97d4-f291d2644237 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:08.794042 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ce16e-8083-4eba-8c76-6e6d4ce28ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:08.794928 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ce16e-8083-4eba-8c76-6e6d4ce28ad3 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.795267 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ce16e-8083-4eba-8c76-6e6d4ce28ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.795553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ce16e-8083-4eba-8c76-6e6d4ce28ad3 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.798154 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ce16e-8083-4eba-8c76-6e6d4ce28ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:08.798896 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8ad15-d7aa-40de-8cb0-159bab21cd8a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:08.799654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ce16e-8083-4eba-8c76-6e6d4ce28ad3 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:08.801740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 122/261] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:08 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:08.802535 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ce16e-8083-4eba-8c76-6e6d4ce28ad3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:08.845964 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997ebf-14ec-40ba-9c2b-cde23db7427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:08.846832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997ebf-14ec-40ba-9c2b-cde23db7427b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:08.847110 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997ebf-14ec-40ba-9c2b-cde23db7427b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:08.847384 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997ebf-14ec-40ba-9c2b-cde23db7427b None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:08.848586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58997ebf-14ec-40ba-9c2b-cde23db7427b None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:08.850232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58997ebf-14ec-40ba-9c2b-cde23db7427b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:08.852412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 140/262] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:08 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:08.853504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997ebf-14ec-40ba-9c2b-cde23db7427b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:08.966754 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997ebf-14ec-40ba-9c2b-cde23db7427b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:09.005884 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38155f-5bcc-4114-a666-527d5cbc9152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:09.006928 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38155f-5bcc-4114-a666-527d5cbc9152 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:09.007210 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38155f-5bcc-4114-a666-527d5cbc9152 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:09.007483 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38155f-5bcc-4114-a666-527d5cbc9152 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814/users/4e2a0d46e4e541b0b147639282d73c31/roles/77a3914adfe94a78b2474e4d74d2bf57` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:09.009480 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38155f-5bcc-4114-a666-527d5cbc9152 None admin] RBAC: Authorizing `identity:create_grant(project_id=66c8b57a8c8e424baf3afade22505814, user_id=4e2a0d46e4e541b0b147639282d73c31, role_id=77a3914adfe94a78b2474e4d74d2bf57)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:09.011423 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38155f-5bcc-4114-a666-527d5cbc9152 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:09.013618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ce16e-8083-4eba-8c76-6e6d4ce28ad3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:09.027677 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 141/263] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:08 2022] PUT /identity/v3/projects/66c8b57a8c8e424baf3afade22505814/users/4e2a0d46e4e541b0b147639282d73c31/roles/77a3914adfe94a78b2474e4d74d2bf57 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:09.029213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38155f-5bcc-4114-a666-527d5cbc9152 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:09.100549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:09.101279 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:09.101566 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:09.101843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:09.104516 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:09.104843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:09.106309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:09.126060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 123/264] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:09.130867 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:09.222183 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38155f-5bcc-4114-a666-527d5cbc9152 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:09.269531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712dace-47f3-4307-a6dc-69e71a000ceb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:09.305245 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a64ea-0f89-46be-bfb7-4bc12c0b09ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:09.306734 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a64ea-0f89-46be-bfb7-4bc12c0b09ee None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:09.307020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a64ea-0f89-46be-bfb7-4bc12c0b09ee None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:09.307300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a64ea-0f89-46be-bfb7-4bc12c0b09ee None admin] PATH_INFO: `/v3/users/24fadea75260432a92706c933c9cbfa6` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:09.308548 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a64ea-0f89-46be-bfb7-4bc12c0b09ee None admin] RBAC: Authorizing `identity:get_user(user_id=24fadea75260432a92706c933c9cbfa6)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:09.310099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a64ea-0f89-46be-bfb7-4bc12c0b09ee None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:09.312115 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 124/265] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:09 2022] GET /identity/v3/users/24fadea75260432a92706c933c9cbfa6 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:09.320259 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:09.320907 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a64ea-0f89-46be-bfb7-4bc12c0b09ee None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:09.321170 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:09.321366 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:09.321614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:09.323057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=66c8b57a8c8e424baf3afade22505814, user.id=24fadea75260432a92706c933c9cbfa6) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:09.323370 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:09.324859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:09.343493 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 142/266] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:09 2022] GET /identity/v3/role_assignments?user.id=24fadea75260432a92706c933c9cbfa6&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:09.464171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a64ea-0f89-46be-bfb7-4bc12c0b09ee None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:09.501650 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b76b-913e-481a-89d7-a4a755daceaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:09.502317 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b76b-913e-481a-89d7-a4a755daceaa None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:09.502568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b76b-913e-481a-89d7-a4a755daceaa None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:09.502847 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b76b-913e-481a-89d7-a4a755daceaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:09.574819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615b76b-913e-481a-89d7-a4a755daceaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:09.576304 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615b76b-913e-481a-89d7-a4a755daceaa None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:09.680547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 125/267] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:09.690187 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:09.822221 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650360b-23a1-4e85-b286-3564c02ad918 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:09.872340 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489824a-7233-438d-ac6d-1d579071f273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:09.873234 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489824a-7233-438d-ac6d-1d579071f273 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:09.873517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489824a-7233-438d-ac6d-1d579071f273 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:09.873792 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489824a-7233-438d-ac6d-1d579071f273 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:09.874999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489824a-7233-438d-ac6d-1d579071f273 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:09.876588 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489824a-7233-438d-ac6d-1d579071f273 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:09.878619 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 143/268] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:09 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:09.884347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b76b-913e-481a-89d7-a4a755daceaa None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:10.004507 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b76b-913e-481a-89d7-a4a755daceaa None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:10.048140 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844829b-c507-4be3-ae72-1fe98e407e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:10.049204 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844829b-c507-4be3-ae72-1fe98e407e58 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.049700 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844829b-c507-4be3-ae72-1fe98e407e58 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.050199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844829b-c507-4be3-ae72-1fe98e407e58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.051263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844829b-c507-4be3-ae72-1fe98e407e58 None admin] RBAC: Adding query filter params (role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=66c8b57a8c8e424baf3afade22505814, user.id=24fadea75260432a92706c933c9cbfa6) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:10.051797 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844829b-c507-4be3-ae72-1fe98e407e58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:10.053491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844829b-c507-4be3-ae72-1fe98e407e58 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:10.060570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 126/269] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:09 2022] GET /identity/v3/role_assignments?user.id=24fadea75260432a92706c933c9cbfa6&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=c8ee1507247e4b34a213a8033c46378f => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:10.566563 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f1579-7706-4316-9fe1-2f4a8894fcb0 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.567880 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f1579-7706-4316-9fe1-2f4a8894fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.568603 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f1579-7706-4316-9fe1-2f4a8894fcb0 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.571414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 144/270] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:10.577254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16ff7b-1bbd-40c6-9696-4172b8f874b7 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.577874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16ff7b-1bbd-40c6-9696-4172b8f874b7 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.577874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16ff7b-1bbd-40c6-9696-4172b8f874b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.596063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-fe16ff7b-1bbd-40c6-9696-4172b8f874b7 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:10.597277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-fe16ff7b-1bbd-40c6-9696-4172b8f874b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:10.641959 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321410b-a73a-4eac-a959-31861dd9c1e4 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.642630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321410b-a73a-4eac-a959-31861dd9c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.643122 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321410b-a73a-4eac-a959-31861dd9c1e4 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.644727 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 145/271] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:10.651102 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ba43-7aa5-4190-8cab-15dab004b0c9 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.651457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ba43-7aa5-4190-8cab-15dab004b0c9 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.651655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1ba43-7aa5-4190-8cab-15dab004b0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.668996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-22a1ba43-7aa5-4190-8cab-15dab004b0c9 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:10.670252 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-22a1ba43-7aa5-4190-8cab-15dab004b0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:10.687459 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 127/272] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:10.711711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16ff7b-1bbd-40c6-9696-4172b8f874b7 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:10.753078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 146/273] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:10.780025 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1ba43-7aa5-4190-8cab-15dab004b0c9 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:10.844982 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16ff7b-1bbd-40c6-9696-4172b8f874b7 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:10.888694 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929ca73-2b51-4c92-86ba-4a0c0f42ca63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:10.889941 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929ca73-2b51-4c92-86ba-4a0c0f42ca63 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.892396 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929ca73-2b51-4c92-86ba-4a0c0f42ca63 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.893242 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929ca73-2b51-4c92-86ba-4a0c0f42ca63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.899080 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1ba43-7aa5-4190-8cab-15dab004b0c9 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:10.905178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929ca73-2b51-4c92-86ba-4a0c0f42ca63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e929ca73-2b51-4c92-86ba-4a0c0f42ca63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:10.913241 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:10.914900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 128/274] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:10.916987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fdec8-7799-4659-bdbd-9fd7dcb1e3ff None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.920349 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fdec8-7799-4659-bdbd-9fd7dcb1e3ff None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.920349 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fdec8-7799-4659-bdbd-9fd7dcb1e3ff None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.921591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 129/275] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:10.923307 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fdec8-7799-4659-bdbd-9fd7dcb1e3ff None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:10.943773 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b11a8-9b06-48a0-9989-a9883d496880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:10.944575 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b11a8-9b06-48a0-9989-a9883d496880 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.944857 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b11a8-9b06-48a0-9989-a9883d496880 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.945163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b11a8-9b06-48a0-9989-a9883d496880 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.946361 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b11a8-9b06-48a0-9989-a9883d496880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:10.947856 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b11a8-9b06-48a0-9989-a9883d496880 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:10.949670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 147/276] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:10.955236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3323c0-a452-4a95-b575-a414f811257b None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:10.955524 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3323c0-a452-4a95-b575-a414f811257b None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:10.955798 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3323c0-a452-4a95-b575-a414f811257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:10.973009 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-6f3323c0-a452-4a95-b575-a414f811257b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:10.974486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-6f3323c0-a452-4a95-b575-a414f811257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:11.034735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fdec8-7799-4659-bdbd-9fd7dcb1e3ff None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.047957 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 148/277] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:11.047957 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3323c0-a452-4a95-b575-a414f811257b None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:11.080205 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.080880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.081194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.081470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.082214 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:11.082514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:11.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:11.096732 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 130/278] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:11.096732 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:11.140462 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3323c0-a452-4a95-b575-a414f811257b None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.174364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64c27b-904a-4b14-8f91-0dde5199c695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.175031 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64c27b-904a-4b14-8f91-0dde5199c695 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.175392 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64c27b-904a-4b14-8f91-0dde5199c695 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.175672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64c27b-904a-4b14-8f91-0dde5199c695 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.176371 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64c27b-904a-4b14-8f91-0dde5199c695 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:11.177840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64c27b-904a-4b14-8f91-0dde5199c695 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:11.190455 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058ea88-2229-4524-96fd-4e0f47ea68d1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.199642 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 149/279] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:11 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:15:11.201026 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64c27b-904a-4b14-8f91-0dde5199c695 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:11.246854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d83c31-84e7-45ee-b471-a4929d67e0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.247988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d83c31-84e7-45ee-b471-a4929d67e0d5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.248337 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d83c31-84e7-45ee-b471-a4929d67e0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.248680 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d83c31-84e7-45ee-b471-a4929d67e0d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.270260 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d83c31-84e7-45ee-b471-a4929d67e0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d5d83c31-84e7-45ee-b471-a4929d67e0d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:11.281247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:11.282394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:11.283299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 131/280] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:11.292762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d83c31-84e7-45ee-b471-a4929d67e0d5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:11.365911 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64c27b-904a-4b14-8f91-0dde5199c695 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.401853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d83c31-84e7-45ee-b471-a4929d67e0d5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.405532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21316cd3-89aa-47bc-8d12-08a0d70f5cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.406626 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21316cd3-89aa-47bc-8d12-08a0d70f5cdd None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.406910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21316cd3-89aa-47bc-8d12-08a0d70f5cdd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.407192 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21316cd3-89aa-47bc-8d12-08a0d70f5cdd None admin] PATH_INFO: `/v3/users/de2bf9de561841cebb16127e042d91f4` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.408496 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21316cd3-89aa-47bc-8d12-08a0d70f5cdd None admin] RBAC: Authorizing `identity:get_user(user_id=de2bf9de561841cebb16127e042d91f4)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:11.410718 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21316cd3-89aa-47bc-8d12-08a0d70f5cdd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:11.413343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 150/281] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:11 2022] GET /identity/v3/users/de2bf9de561841cebb16127e042d91f4 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:11.419789 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21316cd3-89aa-47bc-8d12-08a0d70f5cdd None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:11.448449 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.449175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.449460 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.449743 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.451321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] RBAC: Adding query filter params (name=member) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:11.451689 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:11.453125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:11.466779 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 132/282] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:11.472985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:11.555214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21316cd3-89aa-47bc-8d12-08a0d70f5cdd None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.593412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c12ce-7b23-476a-84cc-dfa392eaa123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.596140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c12ce-7b23-476a-84cc-dfa392eaa123 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.596670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c12ce-7b23-476a-84cc-dfa392eaa123 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.597216 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c12ce-7b23-476a-84cc-dfa392eaa123 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.598742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c12ce-7b23-476a-84cc-dfa392eaa123 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:11.600436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c12ce-7b23-476a-84cc-dfa392eaa123 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:11.602750 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 151/283] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:11 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:11.606075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf05b5-7b3d-456b-9983-c8e53d413877 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.608157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c12ce-7b23-476a-84cc-dfa392eaa123 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:11.642214 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19acda-98fc-4e25-a4d7-05e08ee9bbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.643299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19acda-98fc-4e25-a4d7-05e08ee9bbeb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.643589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19acda-98fc-4e25-a4d7-05e08ee9bbeb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.643865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19acda-98fc-4e25-a4d7-05e08ee9bbeb None admin] PATH_INFO: `/v3/users/ef57871e6722439494d3ea6d049a9620` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.660928 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19acda-98fc-4e25-a4d7-05e08ee9bbeb None admin] RBAC: Authorizing `identity:get_user(user_id=ef57871e6722439494d3ea6d049a9620)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:11.663313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19acda-98fc-4e25-a4d7-05e08ee9bbeb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:11.665081 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 133/284] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:11 2022] GET /identity/v3/users/ef57871e6722439494d3ea6d049a9620 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:11.673276 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19acda-98fc-4e25-a4d7-05e08ee9bbeb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:11.718405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c12ce-7b23-476a-84cc-dfa392eaa123 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.756269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784799a-98dc-4374-bbdf-286b6b497e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.757225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784799a-98dc-4374-bbdf-286b6b497e46 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.757508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784799a-98dc-4374-bbdf-286b6b497e46 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.757780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784799a-98dc-4374-bbdf-286b6b497e46 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814/users/de2bf9de561841cebb16127e042d91f4/roles/f27561b987b14525a832c7d37ff8c62c` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.761644 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784799a-98dc-4374-bbdf-286b6b497e46 None admin] RBAC: Authorizing `identity:create_grant(project_id=66c8b57a8c8e424baf3afade22505814, user_id=de2bf9de561841cebb16127e042d91f4, role_id=f27561b987b14525a832c7d37ff8c62c)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:11.763133 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784799a-98dc-4374-bbdf-286b6b497e46 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:11.781147 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19acda-98fc-4e25-a4d7-05e08ee9bbeb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:11.786020 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 152/285] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:11 2022] PUT /identity/v3/projects/66c8b57a8c8e424baf3afade22505814/users/de2bf9de561841cebb16127e042d91f4/roles/f27561b987b14525a832c7d37ff8c62c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:15:11.846176 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e8787-b57d-4d8b-94e4-0e9286713f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:11.846854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e8787-b57d-4d8b-94e4-0e9286713f2f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:11.847136 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e8787-b57d-4d8b-94e4-0e9286713f2f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:11.847412 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e8787-b57d-4d8b-94e4-0e9286713f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:11.848264 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e8787-b57d-4d8b-94e4-0e9286713f2f None admin] RBAC: Adding query filter params (role.id=77a3914adfe94a78b2474e4d74d2bf57, user.id=ef57871e6722439494d3ea6d049a9620, scope.system=all) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:11.848577 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e8787-b57d-4d8b-94e4-0e9286713f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:11.850070 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e8787-b57d-4d8b-94e4-0e9286713f2f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:11.861748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 134/286] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:15:11 2022] GET /identity/v3/role_assignments?user.id=ef57871e6722439494d3ea6d049a9620&scope.system=all&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:12.047411 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb36e-57b3-4078-b999-b6ea107e0e05 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:12.047812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb36e-57b3-4078-b999-b6ea107e0e05 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:12.048015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb36e-57b3-4078-b999-b6ea107e0e05 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:12.050246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 153/287] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:12.057179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4d1d-4d6c-4deb-bf79-187e3a23c621 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:12.057503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4d1d-4d6c-4deb-bf79-187e3a23c621 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:12.057726 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4d1d-4d6c-4deb-bf79-187e3a23c621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:12.075648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-18cf4d1d-4d6c-4deb-bf79-187e3a23c621 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:12.076841 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-18cf4d1d-4d6c-4deb-bf79-187e3a23c621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:12.147820 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 135/288] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:12.196238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbb36e-57b3-4078-b999-b6ea107e0e05 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:12.332711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbb36e-57b3-4078-b999-b6ea107e0e05 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:12.366290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5395f7-7f58-414d-a2b2-e37fc0631e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:12.367085 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5395f7-7f58-414d-a2b2-e37fc0631e48 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:12.367368 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5395f7-7f58-414d-a2b2-e37fc0631e48 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:12.367643 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5395f7-7f58-414d-a2b2-e37fc0631e48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:12.378329 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5395f7-7f58-414d-a2b2-e37fc0631e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-ec5395f7-7f58-414d-a2b2-e37fc0631e48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:12.386651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:12.387672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:12.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 154/289] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:12.393755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf4d1d-4d6c-4deb-bf79-187e3a23c621 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:12.511949 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf4d1d-4d6c-4deb-bf79-187e3a23c621 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:12.552530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:12.553236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:12.553521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:12.553800 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:12.554557 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] RBAC: Adding query filter params (name=member) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:12.554866 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:12.556297 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:12.564944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 136/290] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:12.570248 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5395f7-7f58-414d-a2b2-e37fc0631e48 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:12.692008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5395f7-7f58-414d-a2b2-e37fc0631e48 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:12.736672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa71212-88ec-466d-a334-4327dfe15b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:12.737587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71212-88ec-466d-a334-4327dfe15b38 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:12.737870 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71212-88ec-466d-a334-4327dfe15b38 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:12.738147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71212-88ec-466d-a334-4327dfe15b38 None admin] PATH_INFO: `/v3/users/4e2a0d46e4e541b0b147639282d73c31` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:12.739452 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa71212-88ec-466d-a334-4327dfe15b38 None admin] RBAC: Authorizing `identity:get_user(user_id=4e2a0d46e4e541b0b147639282d73c31)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:12.740911 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa71212-88ec-466d-a334-4327dfe15b38 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:12.742592 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 155/291] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:12 2022] GET /identity/v3/users/4e2a0d46e4e541b0b147639282d73c31 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:12.749378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:12.868248 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e9e53-9c66-45f4-9386-16df957f6e0c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:12.908502 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf9fe-8e79-4f16-8dac-47027951fae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:12.909196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf9fe-8e79-4f16-8dac-47027951fae1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:12.909592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf9fe-8e79-4f16-8dac-47027951fae1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:12.909748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf9fe-8e79-4f16-8dac-47027951fae1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:12.987372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cf9fe-8e79-4f16-8dac-47027951fae1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:12.988766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cf9fe-8e79-4f16-8dac-47027951fae1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:13.091583 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 137/292] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:13.097080 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa71212-88ec-466d-a334-4327dfe15b38 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:13.232579 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa71212-88ec-466d-a334-4327dfe15b38 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:13.281750 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bb4a5-7c7b-46ec-92e6-cc7d1cc04b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:13.282625 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bb4a5-7c7b-46ec-92e6-cc7d1cc04b13 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:13.282908 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bb4a5-7c7b-46ec-92e6-cc7d1cc04b13 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:13.283184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bb4a5-7c7b-46ec-92e6-cc7d1cc04b13 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:13.284424 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bb4a5-7c7b-46ec-92e6-cc7d1cc04b13 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:13.285901 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bb4a5-7c7b-46ec-92e6-cc7d1cc04b13 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:13.287930 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 156/293] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:13 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:13.298134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf9fe-8e79-4f16-8dac-47027951fae1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:13.500022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf9fe-8e79-4f16-8dac-47027951fae1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:13.568101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51935336-564a-465f-8bba-422ad5271b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:13.568727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51935336-564a-465f-8bba-422ad5271b48 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:13.569013 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51935336-564a-465f-8bba-422ad5271b48 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:13.574271 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51935336-564a-465f-8bba-422ad5271b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:13.575192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51935336-564a-465f-8bba-422ad5271b48 None admin] RBAC: Adding query filter params (role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=66c8b57a8c8e424baf3afade22505814, user.id=4e2a0d46e4e541b0b147639282d73c31) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:13.575519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51935336-564a-465f-8bba-422ad5271b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:13.577020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51935336-564a-465f-8bba-422ad5271b48 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:13.584058 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 138/294] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:13 2022] GET /identity/v3/role_assignments?user.id=4e2a0d46e4e541b0b147639282d73c31&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:14.199397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ec3c0-0a3d-4074-8912-8dc94a1ae273 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.199924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ec3c0-0a3d-4074-8912-8dc94a1ae273 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.200034 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ec3c0-0a3d-4074-8912-8dc94a1ae273 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.201368 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 157/295] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:14.207037 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5501fd-7562-4a73-8ca1-7f3813444c8f None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.207505 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5501fd-7562-4a73-8ca1-7f3813444c8f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.207588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5501fd-7562-4a73-8ca1-7f3813444c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.227100 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-db5501fd-7562-4a73-8ca1-7f3813444c8f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:14.227293 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-db5501fd-7562-4a73-8ca1-7f3813444c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:14.238763 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2ce5-b9a2-46ed-a847-9ca5a10e226b None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.239050 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2ce5-b9a2-46ed-a847-9ca5a10e226b None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.239334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2ce5-b9a2-46ed-a847-9ca5a10e226b None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.240990 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 158/296] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:14.249074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a018-1519-41c9-9607-ab756bdbd129 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.249074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a018-1519-41c9-9607-ab756bdbd129 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.249074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a018-1519-41c9-9607-ab756bdbd129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.274987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-06f5a018-1519-41c9-9607-ab756bdbd129 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:14.276188 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-06f5a018-1519-41c9-9607-ab756bdbd129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:14.323194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 139/297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:14.345962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5501fd-7562-4a73-8ca1-7f3813444c8f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:14.355133 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 159/298] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:14.378736 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a018-1519-41c9-9607-ab756bdbd129 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:14.446687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5501fd-7562-4a73-8ca1-7f3813444c8f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:14.476548 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a018-1519-41c9-9607-ab756bdbd129 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:14.481571 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caad55-87db-4805-a6f7-a2a60e5cc6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:14.482368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caad55-87db-4805-a6f7-a2a60e5cc6c9 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.482647 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caad55-87db-4805-a6f7-a2a60e5cc6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.482921 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caad55-87db-4805-a6f7-a2a60e5cc6c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.484102 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caad55-87db-4805-a6f7-a2a60e5cc6c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:14.485579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caad55-87db-4805-a6f7-a2a60e5cc6c9 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:14.491989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 140/299] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:14.492778 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caad55-87db-4805-a6f7-a2a60e5cc6c9 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:14.511975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30133182-4781-4104-b99c-12c6bfed33b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:14.512771 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30133182-4781-4104-b99c-12c6bfed33b4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.513071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30133182-4781-4104-b99c-12c6bfed33b4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.513349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30133182-4781-4104-b99c-12c6bfed33b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.523380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30133182-4781-4104-b99c-12c6bfed33b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-30133182-4781-4104-b99c-12c6bfed33b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:14.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:14.532336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:14.534417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 160/300] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:14.542035 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30133182-4781-4104-b99c-12c6bfed33b4 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:14.596173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caad55-87db-4805-a6f7-a2a60e5cc6c9 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:14.632426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e0aac-d84f-4148-a73e-9d70057081d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:14.633428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e0aac-d84f-4148-a73e-9d70057081d3 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.633712 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e0aac-d84f-4148-a73e-9d70057081d3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.633984 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e0aac-d84f-4148-a73e-9d70057081d3 None admin] PATH_INFO: `/v3/groups` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.634752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e0aac-d84f-4148-a73e-9d70057081d3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:14.636193 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e0aac-d84f-4148-a73e-9d70057081d3 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:14.649566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30133182-4781-4104-b99c-12c6bfed33b4 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:14.666047 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 141/301] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 9 15:15:14 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:15:14.691743 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049f141-45b9-4063-ae8b-f41f3cc9acd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:14.692434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049f141-45b9-4063-ae8b-f41f3cc9acd6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.692713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049f141-45b9-4063-ae8b-f41f3cc9acd6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.692982 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049f141-45b9-4063-ae8b-f41f3cc9acd6 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.693772 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049f141-45b9-4063-ae8b-f41f3cc9acd6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:14.694076 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049f141-45b9-4063-ae8b-f41f3cc9acd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:14.695547 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049f141-45b9-4063-ae8b-f41f3cc9acd6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:14.705827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 161/302] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:14.711368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e0aac-d84f-4148-a73e-9d70057081d3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:14.760993 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5231124-c26a-47e0-8a32-1f33f371f5ce None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.761219 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5231124-c26a-47e0-8a32-1f33f371f5ce None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.761622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5231124-c26a-47e0-8a32-1f33f371f5ce None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.763503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 162/303] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:14.768781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823252c2-f371-4d6d-a5dc-c505ad1a3b50 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.769199 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823252c2-f371-4d6d-a5dc-c505ad1a3b50 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.769555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823252c2-f371-4d6d-a5dc-c505ad1a3b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.792301 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-823252c2-f371-4d6d-a5dc-c505ad1a3b50 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:14.794617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-823252c2-f371-4d6d-a5dc-c505ad1a3b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:14.832566 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e0aac-d84f-4148-a73e-9d70057081d3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:14.869272 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0f128-76be-4e63-8a94-71fb16eb912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:14.870414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0f128-76be-4e63-8a94-71fb16eb912c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.870696 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0f128-76be-4e63-8a94-71fb16eb912c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.870968 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0f128-76be-4e63-8a94-71fb16eb912c None admin] PATH_INFO: `/v3/users/cd4ea4800ec142679835ef1e3eb62e2f` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.878534 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 163/304] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:14.880126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5118e-86ef-4268-a9c5-3f12502c0ada None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.880580 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5118e-86ef-4268-a9c5-3f12502c0ada None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.880860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5118e-86ef-4268-a9c5-3f12502c0ada None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.882861 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 164/305] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:14.886080 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0f128-76be-4e63-8a94-71fb16eb912c None admin] RBAC: Authorizing `identity:get_user(user_id=cd4ea4800ec142679835ef1e3eb62e2f)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:14.887709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0f128-76be-4e63-8a94-71fb16eb912c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:14.890071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e787-050b-4aef-a1f0-6526e9259d83 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:14.890071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e787-050b-4aef-a1f0-6526e9259d83 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:14.890071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e787-050b-4aef-a1f0-6526e9259d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:14.894033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 142/306] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:14 2022] GET /identity/v3/users/cd4ea4800ec142679835ef1e3eb62e2f => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:14.896774 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0f128-76be-4e63-8a94-71fb16eb912c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:14.909742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-9887e787-050b-4aef-a1f0-6526e9259d83 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:14.910636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-9887e787-050b-4aef-a1f0-6526e9259d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:14.980130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 165/307] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:14.980988 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887e787-050b-4aef-a1f0-6526e9259d83 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:15.008626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0f128-76be-4e63-8a94-71fb16eb912c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.043258 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.043837 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.044121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.044395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.045255 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] RBAC: Adding query filter params (role.id=f27561b987b14525a832c7d37ff8c62c, user.id=cd4ea4800ec142679835ef1e3eb62e2f, scope.system=all) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:15.045569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:15.047022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:15.052089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 143/308] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:15:14 2022] GET /identity/v3/role_assignments?user.id=cd4ea4800ec142679835ef1e3eb62e2f&scope.system=all&role.id=f27561b987b14525a832c7d37ff8c62c => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:15.052864 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:15.114431 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887e787-050b-4aef-a1f0-6526e9259d83 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.161820 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6220-aa3b-4e91-ae18-15540e4594c6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.162965 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca82bc-41b8-4977-8005-4fb5c445c454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.165064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca82bc-41b8-4977-8005-4fb5c445c454 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.166164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca82bc-41b8-4977-8005-4fb5c445c454 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.166650 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca82bc-41b8-4977-8005-4fb5c445c454 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.181487 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca82bc-41b8-4977-8005-4fb5c445c454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-58ca82bc-41b8-4977-8005-4fb5c445c454 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:15.192254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:15.193367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:15.194249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:15.201444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 166/309] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:15.203297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca82bc-41b8-4977-8005-4fb5c445c454 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:15.206673 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540c220-7df6-43d2-8840-d354bd8d2633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.208443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540c220-7df6-43d2-8840-d354bd8d2633 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.208994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540c220-7df6-43d2-8840-d354bd8d2633 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.208994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540c220-7df6-43d2-8840-d354bd8d2633 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.223671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540c220-7df6-43d2-8840-d354bd8d2633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-3540c220-7df6-43d2-8840-d354bd8d2633 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:15.234015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:15.241232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:15.242113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 144/310] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:15.243898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540c220-7df6-43d2-8840-d354bd8d2633 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:15.327376 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca82bc-41b8-4977-8005-4fb5c445c454 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.347067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540c220-7df6-43d2-8840-d354bd8d2633 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.361784 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.362810 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.363248 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.363674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.364589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:15.365102 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:15.366655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:15.375789 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 167/311] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:15.381285 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:15.386604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.387278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.387561 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.387833 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.388584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:15.388890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:15.390300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:15.403604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 145/312] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:15.404501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:15.483444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a6408-6603-4c0f-8723-fb5b21204e11 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.503719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ed0b-8a1c-442e-befe-5b98bd4b2570 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.518023 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107a8ba-59d6-43d0-bf14-2763b1cd0b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.519142 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107a8ba-59d6-43d0-bf14-2763b1cd0b46 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.519579 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107a8ba-59d6-43d0-bf14-2763b1cd0b46 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.520022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107a8ba-59d6-43d0-bf14-2763b1cd0b46 None admin] PATH_INFO: `/v3/users/ef57871e6722439494d3ea6d049a9620` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.521422 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107a8ba-59d6-43d0-bf14-2763b1cd0b46 None admin] RBAC: Authorizing `identity:get_user(user_id=ef57871e6722439494d3ea6d049a9620)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:15.523031 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107a8ba-59d6-43d0-bf14-2763b1cd0b46 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:15.525763 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 168/313] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:15 2022] GET /identity/v3/users/ef57871e6722439494d3ea6d049a9620 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:15.531141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107a8ba-59d6-43d0-bf14-2763b1cd0b46 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:15.544003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376224bc-86d6-44eb-9cac-744bd3524450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.545557 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376224bc-86d6-44eb-9cac-744bd3524450 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.546543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376224bc-86d6-44eb-9cac-744bd3524450 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.547274 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376224bc-86d6-44eb-9cac-744bd3524450 None admin] PATH_INFO: `/v3/users/de2bf9de561841cebb16127e042d91f4` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.548813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376224bc-86d6-44eb-9cac-744bd3524450 None admin] RBAC: Authorizing `identity:get_user(user_id=de2bf9de561841cebb16127e042d91f4)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:15.551135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376224bc-86d6-44eb-9cac-744bd3524450 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:15.553428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 146/314] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:15 2022] GET /identity/v3/users/de2bf9de561841cebb16127e042d91f4 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:15.561252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376224bc-86d6-44eb-9cac-744bd3524450 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:15.659125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107a8ba-59d6-43d0-bf14-2763b1cd0b46 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.678265 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376224bc-86d6-44eb-9cac-744bd3524450 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:15.702276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2d457-762a-4668-9ad7-c4678a943ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.703390 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d457-762a-4668-9ad7-c4678a943ea7 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.703851 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d457-762a-4668-9ad7-c4678a943ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.704287 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d457-762a-4668-9ad7-c4678a943ea7 None admin] PATH_INFO: `/v3/system/users/ef57871e6722439494d3ea6d049a9620/roles/77a3914adfe94a78b2474e4d74d2bf57` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.705967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2d457-762a-4668-9ad7-c4678a943ea7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ef57871e6722439494d3ea6d049a9620, role_id=77a3914adfe94a78b2474e4d74d2bf57)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:15.707582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2d457-762a-4668-9ad7-c4678a943ea7 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:15.718689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 169/315] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 9 15:15:15 2022] PUT /identity/v3/system/users/ef57871e6722439494d3ea6d049a9620/roles/77a3914adfe94a78b2474e4d74d2bf57 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:15.726049 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8994-c2e2-40f0-bfd5-fd19fa057dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:15.727065 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8994-c2e2-40f0-bfd5-fd19fa057dc3 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:15.727579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8994-c2e2-40f0-bfd5-fd19fa057dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:15.728098 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc8994-c2e2-40f0-bfd5-fd19fa057dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:15.814597 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc8994-c2e2-40f0-bfd5-fd19fa057dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:15.816398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc8994-c2e2-40f0-bfd5-fd19fa057dc3 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:15.927709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 147/316] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:15.933635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2d457-762a-4668-9ad7-c4678a943ea7 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:16.041640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2d457-762a-4668-9ad7-c4678a943ea7 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:16.075598 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc24f52-9db3-418b-b10c-674b5a59c5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:16.076813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24f52-9db3-418b-b10c-674b5a59c5c9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:16.077310 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24f52-9db3-418b-b10c-674b5a59c5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:16.077794 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24f52-9db3-418b-b10c-674b5a59c5c9 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:16.079200 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc24f52-9db3-418b-b10c-674b5a59c5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:16.080857 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc24f52-9db3-418b-b10c-674b5a59c5c9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:16.083143 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 170/317] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:15 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:16.088649 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8994-c2e2-40f0-bfd5-fd19fa057dc3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:16.189521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc8994-c2e2-40f0-bfd5-fd19fa057dc3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:16.223979 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f6bf4-3e80-433d-95da-176b9aaad929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:16.224634 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6bf4-3e80-433d-95da-176b9aaad929 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:16.224915 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6bf4-3e80-433d-95da-176b9aaad929 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:16.225222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f6bf4-3e80-433d-95da-176b9aaad929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:16.226054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f6bf4-3e80-433d-95da-176b9aaad929 None admin] RBAC: Adding query filter params (role.id=f27561b987b14525a832c7d37ff8c62c, scope.project.id=66c8b57a8c8e424baf3afade22505814, user.id=de2bf9de561841cebb16127e042d91f4) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:16.226368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f6bf4-3e80-433d-95da-176b9aaad929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:16.228341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f6bf4-3e80-433d-95da-176b9aaad929 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:16.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 148/318] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:16 2022] GET /identity/v3/role_assignments?user.id=de2bf9de561841cebb16127e042d91f4&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=f27561b987b14525a832c7d37ff8c62c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:17.020847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37b4b-1d7a-461b-96d9-d052982e93f3 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:17.021408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37b4b-1d7a-461b-96d9-d052982e93f3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:17.021408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37b4b-1d7a-461b-96d9-d052982e93f3 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:17.022962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 171/319] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:17.028183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8baa-a95f-410e-bfdb-4c36b3bd035b None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:17.028479 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8baa-a95f-410e-bfdb-4c36b3bd035b None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:17.028761 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8baa-a95f-410e-bfdb-4c36b3bd035b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:17.046056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-696a8baa-a95f-410e-bfdb-4c36b3bd035b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:17.047255 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-696a8baa-a95f-410e-bfdb-4c36b3bd035b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:17.110742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 149/320] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:17.134154 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa37b4b-1d7a-461b-96d9-d052982e93f3 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:17.235467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa37b4b-1d7a-461b-96d9-d052982e93f3 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:17.269970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d084f8-c990-46f7-b72e-039092622368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:17.270767 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d084f8-c990-46f7-b72e-039092622368 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:17.271103 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d084f8-c990-46f7-b72e-039092622368 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:17.271327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d084f8-c990-46f7-b72e-039092622368 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:17.272500 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d084f8-c990-46f7-b72e-039092622368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:17.273990 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d084f8-c990-46f7-b72e-039092622368 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:17.276373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 172/321] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:17.281790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a8baa-a95f-410e-bfdb-4c36b3bd035b None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:17.377300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a8baa-a95f-410e-bfdb-4c36b3bd035b None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:17.396959 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2b2a-f846-43dc-8bfb-c4bac0c78e50 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:17.397305 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2b2a-f846-43dc-8bfb-c4bac0c78e50 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:17.397564 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d2b2a-f846-43dc-8bfb-c4bac0c78e50 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:17.399090 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 173/322] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:17.404343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331e6e-3b09-4bd5-a79e-84affad8fe9c None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:17.404629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331e6e-3b09-4bd5-a79e-84affad8fe9c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:17.404907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331e6e-3b09-4bd5-a79e-84affad8fe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:17.412179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf11f93-bb88-4d4c-9627-95607fd1fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:17.412862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf11f93-bb88-4d4c-9627-95607fd1fbc7 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:17.413191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf11f93-bb88-4d4c-9627-95607fd1fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:17.413473 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf11f93-bb88-4d4c-9627-95607fd1fbc7 None admin] PATH_INFO: `/v3/groups` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:17.414179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf11f93-bb88-4d4c-9627-95607fd1fbc7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:17.416197 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf11f93-bb88-4d4c-9627-95607fd1fbc7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:17.426218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-e1331e6e-3b09-4bd5-a79e-84affad8fe9c None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:17.427811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-e1331e6e-3b09-4bd5-a79e-84affad8fe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:17.432524 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 150/323] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 9 15:15:17 2022] POST /identity/v3/groups => generated 226 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:15:17.499771 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 174/324] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:17.521990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf11f93-bb88-4d4c-9627-95607fd1fbc7 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:17.626437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf11f93-bb88-4d4c-9627-95607fd1fbc7 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:17.664691 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bf5a9-bdb4-4544-9a54-b96269fe6f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:17.665566 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bf5a9-bdb4-4544-9a54-b96269fe6f37 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:17.665854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bf5a9-bdb4-4544-9a54-b96269fe6f37 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:17.666129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bf5a9-bdb4-4544-9a54-b96269fe6f37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:17.676453 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bf5a9-bdb4-4544-9a54-b96269fe6f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-aa0bf5a9-bdb4-4544-9a54-b96269fe6f37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:17.684222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:17.685300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:17.686599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 151/325] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:17.691665 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1331e6e-3b09-4bd5-a79e-84affad8fe9c None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:17.797829 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1331e6e-3b09-4bd5-a79e-84affad8fe9c None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:17.835385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:17.836489 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:17.837001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:17.838107 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:17.839190 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] RBAC: Adding query filter params (name=reader) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:17.839739 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:17.841402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:17.851853 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 175/326] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:17.857318 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bf5a9-bdb4-4544-9a54-b96269fe6f37 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:17.970552 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bf5a9-bdb4-4544-9a54-b96269fe6f37 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:18.006007 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4e05-f762-41f5-b3bb-256f0b8da95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:18.007252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4e05-f762-41f5-b3bb-256f0b8da95b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:18.007777 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4e05-f762-41f5-b3bb-256f0b8da95b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:18.008277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4e05-f762-41f5-b3bb-256f0b8da95b None admin] PATH_INFO: `/v3/users/cd4ea4800ec142679835ef1e3eb62e2f` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:18.010358 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc4e05-f762-41f5-b3bb-256f0b8da95b None admin] RBAC: Authorizing `identity:get_user(user_id=cd4ea4800ec142679835ef1e3eb62e2f)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:18.012029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc4e05-f762-41f5-b3bb-256f0b8da95b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:18.014299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 152/327] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:17 2022] GET /identity/v3/users/cd4ea4800ec142679835ef1e3eb62e2f => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:18.019614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:18.025227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b0123-1fd0-41c7-9080-993d7525cf43 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:18.025517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b0123-1fd0-41c7-9080-993d7525cf43 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:18.025826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b0123-1fd0-41c7-9080-993d7525cf43 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:18.027442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 153/328] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:18.034271 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174738d-2ff1-4eaf-b486-22f3225fbd5d None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:18.034556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174738d-2ff1-4eaf-b486-22f3225fbd5d None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:18.034889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174738d-2ff1-4eaf-b486-22f3225fbd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:18.052624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-f174738d-2ff1-4eaf-b486-22f3225fbd5d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:18.053852 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-f174738d-2ff1-4eaf-b486-22f3225fbd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:18.126269 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 154/329] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:18.134600 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ff5ef-fa1b-441d-b494-7192a1ae373b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:18.151462 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174738d-2ff1-4eaf-b486-22f3225fbd5d None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:18.174179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b74156-de0e-47a2-88b2-ac067898d88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:18.175049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b74156-de0e-47a2-88b2-ac067898d88f None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:18.175405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b74156-de0e-47a2-88b2-ac067898d88f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:18.175672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b74156-de0e-47a2-88b2-ac067898d88f None admin] PATH_INFO: `/v3/system/users/cd4ea4800ec142679835ef1e3eb62e2f/roles/f27561b987b14525a832c7d37ff8c62c` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:18.177254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b74156-de0e-47a2-88b2-ac067898d88f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cd4ea4800ec142679835ef1e3eb62e2f, role_id=f27561b987b14525a832c7d37ff8c62c)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:18.178644 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b74156-de0e-47a2-88b2-ac067898d88f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:18.199921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 176/330] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 9 15:15:18 2022] PUT /identity/v3/system/users/cd4ea4800ec142679835ef1e3eb62e2f/roles/f27561b987b14525a832c7d37ff8c62c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:18.271442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174738d-2ff1-4eaf-b486-22f3225fbd5d None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:18.314992 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01105a-a510-4805-bfb0-3a9f6875ee88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:18.315750 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01105a-a510-4805-bfb0-3a9f6875ee88 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:18.316144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01105a-a510-4805-bfb0-3a9f6875ee88 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:18.316350 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01105a-a510-4805-bfb0-3a9f6875ee88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:18.332254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01105a-a510-4805-bfb0-3a9f6875ee88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-dc01105a-a510-4805-bfb0-3a9f6875ee88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:18.340329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:18.341491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:18.343086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 155/331] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:18.347912 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b74156-de0e-47a2-88b2-ac067898d88f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:18.482593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b74156-de0e-47a2-88b2-ac067898d88f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:18.524691 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:18.525755 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:18.526282 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:18.526783 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:18.527754 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:18.528285 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:18.529951 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:18.547242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 177/332] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:18.552933 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01105a-a510-4805-bfb0-3a9f6875ee88 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:18.675574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01105a-a510-4805-bfb0-3a9f6875ee88 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:18.715160 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae704c-fdd9-4adc-a07d-2e7a56a9d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:18.716056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae704c-fdd9-4adc-a07d-2e7a56a9d706 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:18.716344 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae704c-fdd9-4adc-a07d-2e7a56a9d706 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:18.716622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae704c-fdd9-4adc-a07d-2e7a56a9d706 None admin] PATH_INFO: `/v3/users/ef57871e6722439494d3ea6d049a9620` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:18.717902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae704c-fdd9-4adc-a07d-2e7a56a9d706 None admin] RBAC: Authorizing `identity:get_user(user_id=ef57871e6722439494d3ea6d049a9620)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:18.719379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae704c-fdd9-4adc-a07d-2e7a56a9d706 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:18.721052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 156/333] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:18 2022] GET /identity/v3/users/ef57871e6722439494d3ea6d049a9620 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:18.727399 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:18.843595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fb2e7-7dde-4d44-88e1-a005fa79ffac None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:18.879578 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e78e1-b3af-494a-96d0-ce9f67e1cb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:18.880252 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e78e1-b3af-494a-96d0-ce9f67e1cb8a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:18.880536 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e78e1-b3af-494a-96d0-ce9f67e1cb8a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:18.880811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e78e1-b3af-494a-96d0-ce9f67e1cb8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:18.881702 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e78e1-b3af-494a-96d0-ce9f67e1cb8a None admin] RBAC: Adding query filter params (role.id=77a3914adfe94a78b2474e4d74d2bf57, user.id=ef57871e6722439494d3ea6d049a9620, scope.system=all) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:18.882015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e78e1-b3af-494a-96d0-ce9f67e1cb8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:18.883481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e78e1-b3af-494a-96d0-ce9f67e1cb8a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:18.896976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 178/334] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:15:18 2022] GET /identity/v3/role_assignments?user.id=ef57871e6722439494d3ea6d049a9620&scope.system=all&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:20.371338 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f1373-1925-4f96-834f-d512c57e0654 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:20.372030 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f1373-1925-4f96-834f-d512c57e0654 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:20.372030 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f1373-1925-4f96-834f-d512c57e0654 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:20.373464 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 157/335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:20.387168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb804fb-3013-440a-8160-8ff8210d9b83 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:20.387782 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb804fb-3013-440a-8160-8ff8210d9b83 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:20.387782 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb804fb-3013-440a-8160-8ff8210d9b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:20.404775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-0eb804fb-3013-440a-8160-8ff8210d9b83 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:20.406011 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-0eb804fb-3013-440a-8160-8ff8210d9b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:20.479528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 179/336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:20.504982 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f1373-1925-4f96-834f-d512c57e0654 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:20.617198 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f1373-1925-4f96-834f-d512c57e0654 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:20.652263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0271f0d-9311-4e03-997d-df3aff4bdf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:20.653092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271f0d-9311-4e03-997d-df3aff4bdf2a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:20.653376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271f0d-9311-4e03-997d-df3aff4bdf2a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:20.653658 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271f0d-9311-4e03-997d-df3aff4bdf2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:20.664386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0271f0d-9311-4e03-997d-df3aff4bdf2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e0271f0d-9311-4e03-997d-df3aff4bdf2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:20.672760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:20.675145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:20.677287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 158/337] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:20.682889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb804fb-3013-440a-8160-8ff8210d9b83 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:20.782860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb804fb-3013-440a-8160-8ff8210d9b83 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:20.818113 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81affbf9-7a5b-4425-a3a3-09750ba00024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:20.818791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81affbf9-7a5b-4425-a3a3-09750ba00024 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:20.819073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81affbf9-7a5b-4425-a3a3-09750ba00024 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:20.819353 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81affbf9-7a5b-4425-a3a3-09750ba00024 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:20.820111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81affbf9-7a5b-4425-a3a3-09750ba00024 None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:20.820417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81affbf9-7a5b-4425-a3a3-09750ba00024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:20.821966 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81affbf9-7a5b-4425-a3a3-09750ba00024 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:20.831057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 180/338] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:20.836552 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0271f0d-9311-4e03-997d-df3aff4bdf2a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:20.955727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0271f0d-9311-4e03-997d-df3aff4bdf2a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:20.994837 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff6ee6-016f-4e0b-8df7-45c7b7a1ef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:20.995574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff6ee6-016f-4e0b-8df7-45c7b7a1ef86 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:20.995864 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff6ee6-016f-4e0b-8df7-45c7b7a1ef86 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:20.996144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff6ee6-016f-4e0b-8df7-45c7b7a1ef86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.070970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff6ee6-016f-4e0b-8df7-45c7b7a1ef86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:21.072693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff6ee6-016f-4e0b-8df7-45c7b7a1ef86 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:21.114978 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904b43c-e55b-4326-9d76-20d8cfdfa0de None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.115357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904b43c-e55b-4326-9d76-20d8cfdfa0de None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.115523 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904b43c-e55b-4326-9d76-20d8cfdfa0de None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.117071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 181/339] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:21.123849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc826c2d-9c43-4017-bba1-0d9f63036741 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.123849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc826c2d-9c43-4017-bba1-0d9f63036741 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.123849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc826c2d-9c43-4017-bba1-0d9f63036741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.160956 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-bc826c2d-9c43-4017-bba1-0d9f63036741 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:21.162533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-bc826c2d-9c43-4017-bba1-0d9f63036741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:21.187296 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 159/340] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:21.193165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff6ee6-016f-4e0b-8df7-45c7b7a1ef86 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:21.313974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff6ee6-016f-4e0b-8df7-45c7b7a1ef86 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:21.353153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e98d66-0641-41bc-b868-7e0211ff7c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:21.354401 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e98d66-0641-41bc-b868-7e0211ff7c88 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.354866 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e98d66-0641-41bc-b868-7e0211ff7c88 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.355319 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e98d66-0641-41bc-b868-7e0211ff7c88 None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.356691 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e98d66-0641-41bc-b868-7e0211ff7c88 None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:21.358346 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e98d66-0641-41bc-b868-7e0211ff7c88 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:21.358467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 182/341] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:21.360684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 160/342] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:21 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:21.366309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc826c2d-9c43-4017-bba1-0d9f63036741 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:21.422994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e98d66-0641-41bc-b868-7e0211ff7c88 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:21.479900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc826c2d-9c43-4017-bba1-0d9f63036741 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:21.525950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba738ef-39c2-44b0-8726-4519df191da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:21.527179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba738ef-39c2-44b0-8726-4519df191da1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.527684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba738ef-39c2-44b0-8726-4519df191da1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.528159 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba738ef-39c2-44b0-8726-4519df191da1 None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.539541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba738ef-39c2-44b0-8726-4519df191da1 None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:21.541469 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba738ef-39c2-44b0-8726-4519df191da1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:21.543845 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 183/343] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:21 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:21.550035 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba738ef-39c2-44b0-8726-4519df191da1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:21.598618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e98d66-0641-41bc-b868-7e0211ff7c88 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:21.650499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ec78a-7f08-4c4b-96e7-251dc7db3842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:21.651653 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ec78a-7f08-4c4b-96e7-251dc7db3842 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.652109 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ec78a-7f08-4c4b-96e7-251dc7db3842 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.652577 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ec78a-7f08-4c4b-96e7-251dc7db3842 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.670221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ec78a-7f08-4c4b-96e7-251dc7db3842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:21.675390 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba738ef-39c2-44b0-8726-4519df191da1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-837ec78a-7f08-4c4b-96e7-251dc7db3842 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:21.683254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:21.684375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:21.685402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:21.688249 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 161/344] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:21.689876 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d417c7-ebd0-40bf-b65f-961509ea3db8 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.690340 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d417c7-ebd0-40bf-b65f-961509ea3db8 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.690790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d417c7-ebd0-40bf-b65f-961509ea3db8 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.692520 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 162/345] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:21.698425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37edcb9-9738-43be-810c-f35f7b91ddf3 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.698925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37edcb9-9738-43be-810c-f35f7b91ddf3 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.699371 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37edcb9-9738-43be-810c-f35f7b91ddf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.711272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:21.712627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.712627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.712627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.713389 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] RBAC: Adding query filter params (group.id=28407191aa01492d8ce871e63345bb94, role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=84caf63eb1634db7822fe78409be9c77) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:21.713706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:21.715165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:21.725277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 184/346] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:21 2022] GET /identity/v3/role_assignments?group.id=28407191aa01492d8ce871e63345bb94&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:21.726247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:21.730035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-e37edcb9-9738-43be-810c-f35f7b91ddf3 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:21.731421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-e37edcb9-9738-43be-810c-f35f7b91ddf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:21.820755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 163/347] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:21.823099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df12af-7095-4269-a523-3568e8d166d0 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.823570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df12af-7095-4269-a523-3568e8d166d0 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.824006 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df12af-7095-4269-a523-3568e8d166d0 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.825874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 164/348] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:21.838668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88761fff-e762-469a-921a-9642bfe72cf3 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.839036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88761fff-e762-469a-921a-9642bfe72cf3 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.839218 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88761fff-e762-469a-921a-9642bfe72cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.859657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-88761fff-e762-469a-921a-9642bfe72cf3 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:21.860875 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-88761fff-e762-469a-921a-9642bfe72cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:21.867431 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280718b-d532-4491-a103-b6a8b6f11786 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:21.943484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:21.944173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:21.944459 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:21.944743 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:21.945542 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:21.945856 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:21.947836 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:21.953309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 165/349] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:21.954211 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88761fff-e762-469a-921a-9642bfe72cf3 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:21.965583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 185/350] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:21.970830 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:22.120016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88761fff-e762-469a-921a-9642bfe72cf3 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:22.147812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d410a4f-cac6-46eb-8bf0-2cb41fc9e938 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:22.174642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a6b4a-c4dd-47c5-a2c1-8566bbe9d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:22.176124 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a6b4a-c4dd-47c5-a2c1-8566bbe9d1df None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.176651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a6b4a-c4dd-47c5-a2c1-8566bbe9d1df None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.177196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a6b4a-c4dd-47c5-a2c1-8566bbe9d1df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:22.200133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a6b4a-c4dd-47c5-a2c1-8566bbe9d1df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:22.203324 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451dd43-f36f-45db-88ca-313007b39506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:22.204267 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451dd43-f36f-45db-88ca-313007b39506 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.204817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451dd43-f36f-45db-88ca-313007b39506 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.205332 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451dd43-f36f-45db-88ca-313007b39506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-823a6b4a-c4dd-47c5-a2c1-8566bbe9d1df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:22.212568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:22.213687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:22.214588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 166/351] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:22.215396 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a6b4a-c4dd-47c5-a2c1-8566bbe9d1df None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:22.332624 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451dd43-f36f-45db-88ca-313007b39506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:22.334157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451dd43-f36f-45db-88ca-313007b39506 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:22.411156 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a6b4a-c4dd-47c5-a2c1-8566bbe9d1df None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:22.458748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333809c-0b3c-4462-a8d8-25061afc5281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:22.459561 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333809c-0b3c-4462-a8d8-25061afc5281 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.459842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333809c-0b3c-4462-a8d8-25061afc5281 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.460112 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333809c-0b3c-4462-a8d8-25061afc5281 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:22.469099 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 186/352] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:15:22.470048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451dd43-f36f-45db-88ca-313007b39506 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:22.478913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3333809c-0b3c-4462-a8d8-25061afc5281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-3333809c-0b3c-4462-a8d8-25061afc5281 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:22.490887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:22.491957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:22.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 167/353] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:22.495006 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333809c-0b3c-4462-a8d8-25061afc5281 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:22.577529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451dd43-f36f-45db-88ca-313007b39506 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:22.598969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333809c-0b3c-4462-a8d8-25061afc5281 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:22.613113 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:22.613883 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.614172 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.614447 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:22.615207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:22.615514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:22.616894 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:22.627246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 187/354] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:22.628237 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:22.642801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141ed8-9d81-4f96-9840-c9d614d4c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:22.643676 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141ed8-9d81-4f96-9840-c9d614d4c9ca None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.643999 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141ed8-9d81-4f96-9840-c9d614d4c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.644220 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141ed8-9d81-4f96-9840-c9d614d4c9ca None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:22.645475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141ed8-9d81-4f96-9840-c9d614d4c9ca None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:22.646934 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40141ed8-9d81-4f96-9840-c9d614d4c9ca None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:22.648619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 168/355] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:22 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:22.649985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141ed8-9d81-4f96-9840-c9d614d4c9ca None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:22.754219 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40141ed8-9d81-4f96-9840-c9d614d4c9ca None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:22.757621 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597925f5-f0d9-40ca-b846-e14bae8e2dad None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:22.794273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce75a32-5979-47e8-a14d-07c5215aedf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:22.794926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce75a32-5979-47e8-a14d-07c5215aedf1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.795207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce75a32-5979-47e8-a14d-07c5215aedf1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.795482 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce75a32-5979-47e8-a14d-07c5215aedf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:22.803430 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7034a-14c7-43ee-86e5-22808e48b781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:22.804103 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7034a-14c7-43ee-86e5-22808e48b781 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.804392 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7034a-14c7-43ee-86e5-22808e48b781 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.805093 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7034a-14c7-43ee-86e5-22808e48b781 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:22.805911 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7034a-14c7-43ee-86e5-22808e48b781 None admin] RBAC: Adding query filter params (name=reader) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:22.806227 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7034a-14c7-43ee-86e5-22808e48b781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:22.808320 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7034a-14c7-43ee-86e5-22808e48b781 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:22.823268 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 188/356] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:22.824616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c2298-ce39-4f03-ad51-24a41ef322c9 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.824908 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c2298-ce39-4f03-ad51-24a41ef322c9 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.827970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c2298-ce39-4f03-ad51-24a41ef322c9 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:22.829355 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 189/357] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:22.830079 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c2298-ce39-4f03-ad51-24a41ef322c9 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:22.881135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce75a32-5979-47e8-a14d-07c5215aedf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:22.882613 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce75a32-5979-47e8-a14d-07c5215aedf1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:22.958406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c2298-ce39-4f03-ad51-24a41ef322c9 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:22.997808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142ff39-2d57-4aab-9de1-c2d73443f944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:22.998661 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ff39-2d57-4aab-9de1-c2d73443f944 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:22.998936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ff39-2d57-4aab-9de1-c2d73443f944 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:22.999204 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142ff39-2d57-4aab-9de1-c2d73443f944 None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.001504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 169/358] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:23.002357 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce75a32-5979-47e8-a14d-07c5215aedf1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.002473 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142ff39-2d57-4aab-9de1-c2d73443f944 None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:23.004195 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142ff39-2d57-4aab-9de1-c2d73443f944 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:23.008628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 190/359] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:22 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:23.010213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35047-312d-430d-8397-29ae27d0d638 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.010501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35047-312d-430d-8397-29ae27d0d638 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.010781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35047-312d-430d-8397-29ae27d0d638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.031675 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-f7b35047-312d-430d-8397-29ae27d0d638 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:23.032865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-f7b35047-312d-430d-8397-29ae27d0d638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:23.111298 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 191/360] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:23.112655 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce75a32-5979-47e8-a14d-07c5215aedf1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.112910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d24c-3e6d-41cc-8377-744e11bfed2c None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.113232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d24c-3e6d-41cc-8377-744e11bfed2c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.113512 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003d24c-3e6d-41cc-8377-744e11bfed2c None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.119705 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 192/361] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:23.119705 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003d24c-3e6d-41cc-8377-744e11bfed2c None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.160468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d33c48-0abc-4754-a661-ba2baaf50957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.162815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33c48-0abc-4754-a661-ba2baaf50957 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.162815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33c48-0abc-4754-a661-ba2baaf50957 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.162815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33c48-0abc-4754-a661-ba2baaf50957 None admin] PATH_INFO: `/v3/users/cd4ea4800ec142679835ef1e3eb62e2f` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.165557 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d33c48-0abc-4754-a661-ba2baaf50957 None admin] RBAC: Authorizing `identity:get_user(user_id=cd4ea4800ec142679835ef1e3eb62e2f)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:23.167031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d33c48-0abc-4754-a661-ba2baaf50957 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:23.168885 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 170/362] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/users/cd4ea4800ec142679835ef1e3eb62e2f => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:23.173071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d33c48-0abc-4754-a661-ba2baaf50957 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.239678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003d24c-3e6d-41cc-8377-744e11bfed2c None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.293086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d33c48-0abc-4754-a661-ba2baaf50957 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.297219 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ab52a-8fe8-46e9-9239-ea2b134f5b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.299119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ab52a-8fe8-46e9-9239-ea2b134f5b8a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.299574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ab52a-8fe8-46e9-9239-ea2b134f5b8a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.300006 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ab52a-8fe8-46e9-9239-ea2b134f5b8a None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.302779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ab52a-8fe8-46e9-9239-ea2b134f5b8a None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:23.305229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ab52a-8fe8-46e9-9239-ea2b134f5b8a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:23.306118 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 193/363] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:23.307315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54f223-ef8b-4ec6-88f9-5fdee3e4ffcb None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.307783 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54f223-ef8b-4ec6-88f9-5fdee3e4ffcb None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.307902 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed54f223-ef8b-4ec6-88f9-5fdee3e4ffcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.332983 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-ed54f223-ef8b-4ec6-88f9-5fdee3e4ffcb None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:23.334264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-ed54f223-ef8b-4ec6-88f9-5fdee3e4ffcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:23.344673 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.345028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.345685 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.346350 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] RBAC: Adding query filter params (group.id=28407191aa01492d8ce871e63345bb94, role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=66c8b57a8c8e424baf3afade22505814) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:23.346662 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:23.348089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:23.354857 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 171/364] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/role_assignments?group.id=28407191aa01492d8ce871e63345bb94&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:23.355987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.418363 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 194/365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:23.419665 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54f223-ef8b-4ec6-88f9-5fdee3e4ffcb None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.471480 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12153c3c-9d2d-4e0a-a298-1e44f4e5cf6b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.508632 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f791f-1c92-42ac-b3c0-b5ed00051149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.509861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f791f-1c92-42ac-b3c0-b5ed00051149 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.510293 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f791f-1c92-42ac-b3c0-b5ed00051149 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.510743 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f791f-1c92-42ac-b3c0-b5ed00051149 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.522658 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f791f-1c92-42ac-b3c0-b5ed00051149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:23.527713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed54f223-ef8b-4ec6-88f9-5fdee3e4ffcb None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-013f791f-1c92-42ac-b3c0-b5ed00051149 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:23.532087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:23.533330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:23.534475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:23.537022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 172/366] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:23.541821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f791f-1c92-42ac-b3c0-b5ed00051149 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.571214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.572452 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.572813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.573523 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.574127 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] RBAC: Adding query filter params (role.id=f27561b987b14525a832c7d37ff8c62c, user.id=cd4ea4800ec142679835ef1e3eb62e2f, scope.system=all) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:23.574539 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:23.576372 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:23.590424 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 195/367] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/role_assignments?user.id=cd4ea4800ec142679835ef1e3eb62e2f&scope.system=all&role.id=f27561b987b14525a832c7d37ff8c62c => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:23.591078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.658597 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f791f-1c92-42ac-b3c0-b5ed00051149 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.699666 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9b2b-a900-4420-b13a-3a53645ea25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.700919 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9b2b-a900-4420-b13a-3a53645ea25c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.701421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9b2b-a900-4420-b13a-3a53645ea25c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.701887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9b2b-a900-4420-b13a-3a53645ea25c None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.703314 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d9b2b-a900-4420-b13a-3a53645ea25c None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:23.704835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ddbe0-d856-422c-acd2-65d21e8d96e3 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.705271 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d9b2b-a900-4420-b13a-3a53645ea25c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:23.707611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 173/368] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:23.708696 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9b2b-a900-4420-b13a-3a53645ea25c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.745648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab82cc1-fa56-4a8f-a116-3044d014931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.746633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab82cc1-fa56-4a8f-a116-3044d014931f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.746891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab82cc1-fa56-4a8f-a116-3044d014931f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.747265 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab82cc1-fa56-4a8f-a116-3044d014931f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.762824 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab82cc1-fa56-4a8f-a116-3044d014931f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-bab82cc1-fa56-4a8f-a116-3044d014931f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:23.770788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:23.771822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:23.772709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 196/369] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:23.773595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab82cc1-fa56-4a8f-a116-3044d014931f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.822215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9b2b-a900-4420-b13a-3a53645ea25c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.863451 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.864499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.865620 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.866116 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.866972 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:23.867362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:23.870304 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:23.885072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 174/370] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:23.885072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:23.896541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab82cc1-fa56-4a8f-a116-3044d014931f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:23.934733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:23.935495 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:23.935779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:23.936104 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:23.936960 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] RBAC: Adding query filter params (group.id=28407191aa01492d8ce871e63345bb94, role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=84caf63eb1634db7822fe78409be9c77) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:23.937319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:23.938785 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:23.946111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 197/371] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/role_assignments?group.id=28407191aa01492d8ce871e63345bb94&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=c8ee1507247e4b34a213a8033c46378f => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:23.947044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:24.025410 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735d465-3651-4cc2-a251-c8d5a3a8b972 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:24.068290 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:24.069567 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.070250 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.070756 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.072175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:24.073079 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:24.074674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:24.074791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92d722-8e45-4797-986b-e547ef2c9efe None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:24.087404 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 175/372] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:24.092467 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:24.114504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9132e5-c6aa-4ce0-b2aa-fbf6618ebdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:24.115325 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9132e5-c6aa-4ce0-b2aa-fbf6618ebdd7 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.115835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9132e5-c6aa-4ce0-b2aa-fbf6618ebdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.116159 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9132e5-c6aa-4ce0-b2aa-fbf6618ebdd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.212169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9132e5-c6aa-4ce0-b2aa-fbf6618ebdd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:24.214165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9132e5-c6aa-4ce0-b2aa-fbf6618ebdd7 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:24.228536 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f56267-e0d0-4fe6-b075-4b9cf1cd397a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:24.277852 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb26b-a1ad-4dd6-9260-1fe5e1a8a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:24.278635 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb26b-a1ad-4dd6-9260-1fe5e1a8a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.279025 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb26b-a1ad-4dd6-9260-1fe5e1a8a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.279406 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb26b-a1ad-4dd6-9260-1fe5e1a8a0d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.344017 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 198/373] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:24.353925 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9132e5-c6aa-4ce0-b2aa-fbf6618ebdd7 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:24.369509 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb26b-a1ad-4dd6-9260-1fe5e1a8a0d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:24.371449 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb26b-a1ad-4dd6-9260-1fe5e1a8a0d8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:24.501913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 176/374] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:24.508667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb26b-a1ad-4dd6-9260-1fe5e1a8a0d8 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:24.559690 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9132e5-c6aa-4ce0-b2aa-fbf6618ebdd7 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:24.623674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5284-187f-4a9b-b054-5fd215648953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:24.624468 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5284-187f-4a9b-b054-5fd215648953 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.624754 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5284-187f-4a9b-b054-5fd215648953 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.625055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab5284-187f-4a9b-b054-5fd215648953 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.627270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5284-187f-4a9b-b054-5fd215648953 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:24.628740 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab5284-187f-4a9b-b054-5fd215648953 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:24.630505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 199/375] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:24 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:24.643000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5284-187f-4a9b-b054-5fd215648953 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:24.692587 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb26b-a1ad-4dd6-9260-1fe5e1a8a0d8 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:24.734773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31339064-7c07-4ec6-b171-26a594cde980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:24.736649 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339064-7c07-4ec6-b171-26a594cde980 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.737706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339064-7c07-4ec6-b171-26a594cde980 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.738357 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31339064-7c07-4ec6-b171-26a594cde980 None admin] PATH_INFO: `/v3/projects/2f91da8032e14759941fd956b26044bc` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.743695 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31339064-7c07-4ec6-b171-26a594cde980 None admin] RBAC: Authorizing `identity:get_project(project_id=2f91da8032e14759941fd956b26044bc)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:24.745596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31339064-7c07-4ec6-b171-26a594cde980 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:24.748353 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 177/376] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:24 2022] GET /identity/v3/projects/2f91da8032e14759941fd956b26044bc => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:24.750063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e5cb0-0134-4d1b-9c3f-b84e2dfda3ba None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.750063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e5cb0-0134-4d1b-9c3f-b84e2dfda3ba None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.750266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e5cb0-0134-4d1b-9c3f-b84e2dfda3ba None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.751778 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 178/377] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:24.756524 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e5cb0-0134-4d1b-9c3f-b84e2dfda3ba None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:24.780735 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab5284-187f-4a9b-b054-5fd215648953 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:24.817473 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe32b43-b5b2-49e4-95ed-d591e950e8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:24.818702 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe32b43-b5b2-49e4-95ed-d591e950e8e5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.819208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe32b43-b5b2-49e4-95ed-d591e950e8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.819705 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe32b43-b5b2-49e4-95ed-d591e950e8e5 None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.821166 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe32b43-b5b2-49e4-95ed-d591e950e8e5 None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:24.822870 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe32b43-b5b2-49e4-95ed-d591e950e8e5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:24.824991 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 200/378] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:24 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:24.826481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470487-0f26-4d42-b934-46fd76f738a2 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.827003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470487-0f26-4d42-b934-46fd76f738a2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.827501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470487-0f26-4d42-b934-46fd76f738a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.844871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-63470487-0f26-4d42-b934-46fd76f738a2 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:24.846455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-63470487-0f26-4d42-b934-46fd76f738a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:24.871765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e5cb0-0134-4d1b-9c3f-b84e2dfda3ba None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:24.911789 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c5b48-df70-41f9-99bc-c243a8433416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:24.912661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c5b48-df70-41f9-99bc-c243a8433416 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:24.912978 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c5b48-df70-41f9-99bc-c243a8433416 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:24.913289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c5b48-df70-41f9-99bc-c243a8433416 None admin] PATH_INFO: `/v3/groups/7c613ef05cbd4156a8010186ea679681` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:24.917329 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 201/379] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:24.918448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63470487-0f26-4d42-b934-46fd76f738a2 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:24.920328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c5b48-df70-41f9-99bc-c243a8433416 None admin] RBAC: Authorizing `identity:get_group(group_id=7c613ef05cbd4156a8010186ea679681)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:24.922530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c5b48-df70-41f9-99bc-c243a8433416 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:24.923800 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 179/380] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:24 2022] GET /identity/v3/groups/7c613ef05cbd4156a8010186ea679681 => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:24.929427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c5b48-df70-41f9-99bc-c243a8433416 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:25.037199 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63470487-0f26-4d42-b934-46fd76f738a2 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:25.063053 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c5b48-df70-41f9-99bc-c243a8433416 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:25.080974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:25.081601 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:25.081968 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:25.082409 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:25.083284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] RBAC: Adding query filter params (group.id=28407191aa01492d8ce871e63345bb94, role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=66c8b57a8c8e424baf3afade22505814) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:25.083695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:25.085295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:25.093074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 202/381] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:24 2022] GET /identity/v3/role_assignments?group.id=28407191aa01492d8ce871e63345bb94&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=c8ee1507247e4b34a213a8033c46378f => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:25.093865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:25.104295 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:25.105325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:25.106817 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:25.107297 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:25.108353 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] RBAC: Adding query filter params (group.id=7c613ef05cbd4156a8010186ea679681, role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=2f91da8032e14759941fd956b26044bc) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:25.108914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:25.110638 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:25.118101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 180/382] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:24 2022] GET /identity/v3/role_assignments?group.id=7c613ef05cbd4156a8010186ea679681&scope.project.id=2f91da8032e14759941fd956b26044bc&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:25.279214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a68e2a-8f75-462d-87f3-89a074047e18 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:25.325778 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef894a-ab27-4714-836c-29e027907849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:25.326592 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef894a-ab27-4714-836c-29e027907849 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:25.326870 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef894a-ab27-4714-836c-29e027907849 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:25.327140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef894a-ab27-4714-836c-29e027907849 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:25.343932 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef894a-ab27-4714-836c-29e027907849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-feef894a-ab27-4714-836c-29e027907849 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:25.351791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:25.352947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:25.353887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 203/383] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:25.359389 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:25.474466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe9404-08ea-49c3-af02-384997a95927 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:25.513753 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:25.514385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:25.514637 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:25.514879 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:25.515629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] RBAC: Adding query filter params (name=member) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:25.515932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:25.517498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:25.526230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 181/384] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:25.531873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef894a-ab27-4714-836c-29e027907849 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:25.642071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef894a-ab27-4714-836c-29e027907849 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:25.678098 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed35ae3-0ac1-4acf-a963-21a0fedce8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:25.678955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed35ae3-0ac1-4acf-a963-21a0fedce8df None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:25.679293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed35ae3-0ac1-4acf-a963-21a0fedce8df None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:25.679569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed35ae3-0ac1-4acf-a963-21a0fedce8df None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:25.681360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed35ae3-0ac1-4acf-a963-21a0fedce8df None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:25.682801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed35ae3-0ac1-4acf-a963-21a0fedce8df None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:25.684760 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 204/385] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:25 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:25.693028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:25.852025 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35af7ac-e32d-46e4-a93c-b59c235efa5b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:25.888900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff0581-8ba4-4385-8027-ac4bd766abda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:25.889804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff0581-8ba4-4385-8027-ac4bd766abda None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:25.890092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff0581-8ba4-4385-8027-ac4bd766abda None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:25.890368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff0581-8ba4-4385-8027-ac4bd766abda None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:25.891570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff0581-8ba4-4385-8027-ac4bd766abda None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:25.893064 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff0581-8ba4-4385-8027-ac4bd766abda None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:25.895175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 182/386] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:25 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:25.900769 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed35ae3-0ac1-4acf-a963-21a0fedce8df None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:26.047945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed35ae3-0ac1-4acf-a963-21a0fedce8df None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:26.086131 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbeac0-371b-49b4-9b3b-d5291df19513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:26.087314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbeac0-371b-49b4-9b3b-d5291df19513 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:26.087746 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbeac0-371b-49b4-9b3b-d5291df19513 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:26.088168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbeac0-371b-49b4-9b3b-d5291df19513 None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77/groups/28407191aa01492d8ce871e63345bb94/roles/77a3914adfe94a78b2474e4d74d2bf57` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:26.090400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbeac0-371b-49b4-9b3b-d5291df19513 None admin] RBAC: Authorizing `identity:create_grant(project_id=84caf63eb1634db7822fe78409be9c77, group_id=28407191aa01492d8ce871e63345bb94, role_id=77a3914adfe94a78b2474e4d74d2bf57)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:26.092093 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbeac0-371b-49b4-9b3b-d5291df19513 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:26.132069 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 205/387] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:15:25 2022] PUT /identity/v3/projects/84caf63eb1634db7822fe78409be9c77/groups/28407191aa01492d8ce871e63345bb94/roles/77a3914adfe94a78b2474e4d74d2bf57 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:26.737715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de411-253a-4683-83ce-5474a15785f8 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:26.738298 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de411-253a-4683-83ce-5474a15785f8 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:26.738298 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69de411-253a-4683-83ce-5474a15785f8 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:26.739961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 183/388] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:26.746193 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a77e-38be-4f07-9a45-e0dc23775018 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:26.746649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a77e-38be-4f07-9a45-e0dc23775018 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:26.746766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a77e-38be-4f07-9a45-e0dc23775018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:26.766666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-5765a77e-38be-4f07-9a45-e0dc23775018 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:26.768178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-5765a77e-38be-4f07-9a45-e0dc23775018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:26.856014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 206/389] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:26.899071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69de411-253a-4683-83ce-5474a15785f8 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:27.032799 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c692e95-8fff-482c-8c72-298992fe68bb None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.033000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c692e95-8fff-482c-8c72-298992fe68bb None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.033287 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c692e95-8fff-482c-8c72-298992fe68bb None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.034884 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 207/390] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:27.038679 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69de411-253a-4683-83ce-5474a15785f8 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:27.041235 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d803e7-7558-40a1-8a5e-9ea40c9078bd None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.041774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d803e7-7558-40a1-8a5e-9ea40c9078bd None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.042254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d803e7-7558-40a1-8a5e-9ea40c9078bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.078121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-50d803e7-7558-40a1-8a5e-9ea40c9078bd None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:27.079320 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-50d803e7-7558-40a1-8a5e-9ea40c9078bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:27.097957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b1ef2-7765-4566-bc24-4bfcee327da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:27.098765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b1ef2-7765-4566-bc24-4bfcee327da4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.099052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b1ef2-7765-4566-bc24-4bfcee327da4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.099334 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b1ef2-7765-4566-bc24-4bfcee327da4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.122058 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b1ef2-7765-4566-bc24-4bfcee327da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-2a7b1ef2-7765-4566-bc24-4bfcee327da4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:27.130283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:27.131311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:27.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:27.137524 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 184/391] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:27.142883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b1ef2-7765-4566-bc24-4bfcee327da4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:27.174956 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 208/392] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:27.211035 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d803e7-7558-40a1-8a5e-9ea40c9078bd None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:27.343537 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b1ef2-7765-4566-bc24-4bfcee327da4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:27.403327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d803e7-7558-40a1-8a5e-9ea40c9078bd None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:27.431335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:27.432028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.432320 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.432742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.433412 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] RBAC: Adding query filter params (name=member) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:27.433722 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:27.435126 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:27.452724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 185/393] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:27.458162 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:27.461337 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cec29-e2c5-413f-a6a6-104cc129a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:27.462955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cec29-e2c5-413f-a6a6-104cc129a589 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.463513 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cec29-e2c5-413f-a6a6-104cc129a589 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.463984 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cec29-e2c5-413f-a6a6-104cc129a589 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.476468 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cec29-e2c5-413f-a6a6-104cc129a589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-414cec29-e2c5-413f-a6a6-104cc129a589 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:27.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:27.489465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:27.490360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:27.494081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 209/394] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:27.500150 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cec29-e2c5-413f-a6a6-104cc129a589 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:27.624715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cec29-e2c5-413f-a6a6-104cc129a589 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:27.666107 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:27.667189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.667721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.668199 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.669188 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:27.669731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:27.671456 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:27.672329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe87f5-5d51-48b1-90f4-c1c0caedf06c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:27.685577 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 210/395] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:27.691113 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:27.736263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38394d-04dc-4e34-9c2f-3b08a5ab47a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:27.737173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38394d-04dc-4e34-9c2f-3b08a5ab47a1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.737466 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38394d-04dc-4e34-9c2f-3b08a5ab47a1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.737749 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38394d-04dc-4e34-9c2f-3b08a5ab47a1 None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.741791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa38394d-04dc-4e34-9c2f-3b08a5ab47a1 None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:27.743355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa38394d-04dc-4e34-9c2f-3b08a5ab47a1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:27.746428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 186/396] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:27 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:27.750747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38394d-04dc-4e34-9c2f-3b08a5ab47a1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:27.817687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f99eb0-000c-4325-83fa-61e1ac87a1f3 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:27.862831 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfacd2-0a34-4f1c-9dc4-d23a426add68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:27.864184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfacd2-0a34-4f1c-9dc4-d23a426add68 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.864672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfacd2-0a34-4f1c-9dc4-d23a426add68 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.865219 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfacd2-0a34-4f1c-9dc4-d23a426add68 None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.866674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfacd2-0a34-4f1c-9dc4-d23a426add68 None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:27.868368 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfacd2-0a34-4f1c-9dc4-d23a426add68 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:27.870962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 211/397] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:27 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:27.878389 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfacd2-0a34-4f1c-9dc4-d23a426add68 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:27.910726 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38394d-04dc-4e34-9c2f-3b08a5ab47a1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:27.963719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402401d-ae87-41cf-af10-1750e46b1953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:27.963719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402401d-ae87-41cf-af10-1750e46b1953 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:27.964057 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402401d-ae87-41cf-af10-1750e46b1953 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:27.964518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402401d-ae87-41cf-af10-1750e46b1953 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:27.972283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402401d-ae87-41cf-af10-1750e46b1953 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:27.973698 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402401d-ae87-41cf-af10-1750e46b1953 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:27.976020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 187/398] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:27 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:27.983709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402401d-ae87-41cf-af10-1750e46b1953 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:28.023876 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfacd2-0a34-4f1c-9dc4-d23a426add68 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:28.071849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fc84c-e907-4aa1-8b55-66c47ce435aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:28.075044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fc84c-e907-4aa1-8b55-66c47ce435aa None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.075044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fc84c-e907-4aa1-8b55-66c47ce435aa None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.075044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fc84c-e907-4aa1-8b55-66c47ce435aa None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.075583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fc84c-e907-4aa1-8b55-66c47ce435aa None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:28.077191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fc84c-e907-4aa1-8b55-66c47ce435aa None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:28.080287 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 212/399] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:27 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:28.081712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d7db2-41c7-4b9e-9766-00a22c6ed5a5 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.082007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d7db2-41c7-4b9e-9766-00a22c6ed5a5 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.082289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d7db2-41c7-4b9e-9766-00a22c6ed5a5 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.083807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 213/400] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:28.088787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d7db2-41c7-4b9e-9766-00a22c6ed5a5 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:28.119410 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402401d-ae87-41cf-af10-1750e46b1953 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:28.158157 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf72902-f0aa-40a8-a502-ce804a7a2230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:28.159054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf72902-f0aa-40a8-a502-ce804a7a2230 None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.159342 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf72902-f0aa-40a8-a502-ce804a7a2230 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.159621 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf72902-f0aa-40a8-a502-ce804a7a2230 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814/groups/28407191aa01492d8ce871e63345bb94/roles/77a3914adfe94a78b2474e4d74d2bf57` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.162146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf72902-f0aa-40a8-a502-ce804a7a2230 None admin] RBAC: Authorizing `identity:create_grant(project_id=66c8b57a8c8e424baf3afade22505814, group_id=28407191aa01492d8ce871e63345bb94, role_id=77a3914adfe94a78b2474e4d74d2bf57)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:28.163660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf72902-f0aa-40a8-a502-ce804a7a2230 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:28.179690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 188/401] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:15:27 2022] PUT /identity/v3/projects/66c8b57a8c8e424baf3afade22505814/groups/28407191aa01492d8ce871e63345bb94/roles/77a3914adfe94a78b2474e4d74d2bf57 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:15:28.182537 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd97cb-6de8-4bd6-b4a0-ba00c58eae0c None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.183212 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd97cb-6de8-4bd6-b4a0-ba00c58eae0c None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.184015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd97cb-6de8-4bd6-b4a0-ba00c58eae0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-10bd97cb-6de8-4bd6-b4a0-ba00c58eae0c None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:28.207345 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-10bd97cb-6de8-4bd6-b4a0-ba00c58eae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:28.229895 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d7db2-41c7-4b9e-9766-00a22c6ed5a5 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:28.273354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97875-d3fe-4e88-8d14-d68767517f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:28.274408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97875-d3fe-4e88-8d14-d68767517f91 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.274789 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97875-d3fe-4e88-8d14-d68767517f91 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.275131 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97875-d3fe-4e88-8d14-d68767517f91 None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77/groups/28407191aa01492d8ce871e63345bb94/roles/c8ee1507247e4b34a213a8033c46378f` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.277822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df97875-d3fe-4e88-8d14-d68767517f91 None admin] RBAC: Authorizing `identity:create_grant(project_id=84caf63eb1634db7822fe78409be9c77, group_id=28407191aa01492d8ce871e63345bb94, role_id=c8ee1507247e4b34a213a8033c46378f)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:28.279378 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df97875-d3fe-4e88-8d14-d68767517f91 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:28.301419 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 214/402] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:15:28 2022] PUT /identity/v3/projects/84caf63eb1634db7822fe78409be9c77/groups/28407191aa01492d8ce871e63345bb94/roles/c8ee1507247e4b34a213a8033c46378f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:15:28.303174 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40370377-cf0a-4f06-b206-744d870c7777 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.303659 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40370377-cf0a-4f06-b206-744d870c7777 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.304156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40370377-cf0a-4f06-b206-744d870c7777 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.306406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 215/403] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:28.309177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 189/404] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:28.311941 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfdddb-b6b0-43e0-afed-42f098dfb02e None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.312429 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfdddb-b6b0-43e0-afed-42f098dfb02e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.312899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfdddb-b6b0-43e0-afed-42f098dfb02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.327261 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd97cb-6de8-4bd6-b4a0-ba00c58eae0c None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:28.335441 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-5bcfdddb-b6b0-43e0-afed-42f098dfb02e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:28.337238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-5bcfdddb-b6b0-43e0-afed-42f098dfb02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:28.453480 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd97cb-6de8-4bd6-b4a0-ba00c58eae0c None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:28.487885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 216/405] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:28.494777 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca036927-a591-4d77-ad07-1e0f3be6a8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:28.495579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca036927-a591-4d77-ad07-1e0f3be6a8a6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.495863 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca036927-a591-4d77-ad07-1e0f3be6a8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.496138 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca036927-a591-4d77-ad07-1e0f3be6a8a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.513103 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca036927-a591-4d77-ad07-1e0f3be6a8a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-ca036927-a591-4d77-ad07-1e0f3be6a8a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:28.518051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:28.519094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:28.520002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 190/406] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:28.526184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfdddb-b6b0-43e0-afed-42f098dfb02e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:28.530579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca036927-a591-4d77-ad07-1e0f3be6a8a6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:28.650188 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfdddb-b6b0-43e0-afed-42f098dfb02e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:28.650574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca036927-a591-4d77-ad07-1e0f3be6a8a6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:28.694413 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37558ad5-5039-4cd5-83f5-5b884b30776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:28.695230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37558ad5-5039-4cd5-83f5-5b884b30776b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.695509 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37558ad5-5039-4cd5-83f5-5b884b30776b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.695783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37558ad5-5039-4cd5-83f5-5b884b30776b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.716718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37558ad5-5039-4cd5-83f5-5b884b30776b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-37558ad5-5039-4cd5-83f5-5b884b30776b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:28.724592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:28.725606 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:28.726582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:28.727259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 191/407] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:28.734438 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37558ad5-5039-4cd5-83f5-5b884b30776b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:28.735877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:28.736559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:28.736836 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:28.737124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:28.737883 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:28.738191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:28.739572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:28.757063 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 217/408] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:28.757063 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:28.927689 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37558ad5-5039-4cd5-83f5-5b884b30776b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:28.950461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1351f5e-1769-43b1-9b1b-7a231002fd0e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:29.001830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:29.002872 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.003378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.003861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.004828 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:29.005378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:29.007021 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:29.012238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d37f7e-7f67-455c-9345-980bcf9c26de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:29.013320 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d37f7e-7f67-455c-9345-980bcf9c26de None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.013458 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d37f7e-7f67-455c-9345-980bcf9c26de None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.013666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d37f7e-7f67-455c-9345-980bcf9c26de None admin] PATH_INFO: `/v3/groups/7c613ef05cbd4156a8010186ea679681` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.014848 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d37f7e-7f67-455c-9345-980bcf9c26de None admin] RBAC: Authorizing `identity:get_group(group_id=7c613ef05cbd4156a8010186ea679681)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:29.016297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d37f7e-7f67-455c-9345-980bcf9c26de None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:29.017671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 192/409] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:29.022175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 218/410] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:28 2022] GET /identity/v3/groups/7c613ef05cbd4156a8010186ea679681 => generated 229 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:29.023340 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:29.025082 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d37f7e-7f67-455c-9345-980bcf9c26de None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:29.124628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b3ea6-e81e-418e-92c9-5324524b157a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:29.125736 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d37f7e-7f67-455c-9345-980bcf9c26de None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:29.163014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f58a4-302e-46bc-b86a-014390343b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:29.164301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f58a4-302e-46bc-b86a-014390343b81 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.164796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f58a4-302e-46bc-b86a-014390343b81 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.165840 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f58a4-302e-46bc-b86a-014390343b81 None admin] PATH_INFO: `/v3/projects/2f91da8032e14759941fd956b26044bc` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.168020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f58a4-302e-46bc-b86a-014390343b81 None admin] RBAC: Authorizing `identity:get_project(project_id=2f91da8032e14759941fd956b26044bc)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:29.170165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f58a4-302e-46bc-b86a-014390343b81 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:29.172464 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 193/411] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:29 2022] GET /identity/v3/projects/2f91da8032e14759941fd956b26044bc => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:29.175623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0be2f-00ce-4a6b-aff3-ce6d7a7e42a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:29.176741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0be2f-00ce-4a6b-aff3-ce6d7a7e42a4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.177280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0be2f-00ce-4a6b-aff3-ce6d7a7e42a4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.177765 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0be2f-00ce-4a6b-aff3-ce6d7a7e42a4 None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.178751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f58a4-302e-46bc-b86a-014390343b81 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:29.183325 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0be2f-00ce-4a6b-aff3-ce6d7a7e42a4 None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:29.185022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0be2f-00ce-4a6b-aff3-ce6d7a7e42a4 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:29.188550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 219/412] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:29 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:29.209245 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0be2f-00ce-4a6b-aff3-ce6d7a7e42a4 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:29.379251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f58a4-302e-46bc-b86a-014390343b81 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:29.409986 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0be2f-00ce-4a6b-aff3-ce6d7a7e42a4 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:29.424819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db8476-d41e-4171-8d98-94ced6c71c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:29.425738 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8476-d41e-4171-8d98-94ced6c71c24 None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.426023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8476-d41e-4171-8d98-94ced6c71c24 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.426298 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8476-d41e-4171-8d98-94ced6c71c24 None admin] PATH_INFO: `/v3/projects/2f91da8032e14759941fd956b26044bc/groups/7c613ef05cbd4156a8010186ea679681/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.428300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db8476-d41e-4171-8d98-94ced6c71c24 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f91da8032e14759941fd956b26044bc, group_id=7c613ef05cbd4156a8010186ea679681, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:29.429763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db8476-d41e-4171-8d98-94ced6c71c24 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:29.447485 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 194/413] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:15:29 2022] PUT /identity/v3/projects/2f91da8032e14759941fd956b26044bc/groups/7c613ef05cbd4156a8010186ea679681/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:29.456988 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66b284-cb87-468a-bf2b-b2baa5c7e6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:29.458016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66b284-cb87-468a-bf2b-b2baa5c7e6d5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.458301 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66b284-cb87-468a-bf2b-b2baa5c7e6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.458573 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66b284-cb87-468a-bf2b-b2baa5c7e6d5 None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.459981 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66b284-cb87-468a-bf2b-b2baa5c7e6d5 None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:29.461586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66b284-cb87-468a-bf2b-b2baa5c7e6d5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:29.464346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 220/414] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:29 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:29.470146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db8476-d41e-4171-8d98-94ced6c71c24 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:29.535591 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ba490-523f-4d69-8928-726eeef70c1f None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.535905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ba490-523f-4d69-8928-726eeef70c1f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.536125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ba490-523f-4d69-8928-726eeef70c1f None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.537974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 221/415] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:29.554712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2834d-8944-466f-8c85-50308a0ef3cd None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.555064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2834d-8944-466f-8c85-50308a0ef3cd None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.555285 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2834d-8944-466f-8c85-50308a0ef3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.585931 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db8476-d41e-4171-8d98-94ced6c71c24 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:29.590342 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-33b2834d-8944-466f-8c85-50308a0ef3cd None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:29.591622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-33b2834d-8944-466f-8c85-50308a0ef3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:29.641899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08e793-a473-4e64-9404-24c54617743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:29.642825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08e793-a473-4e64-9404-24c54617743d None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.643113 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08e793-a473-4e64-9404-24c54617743d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.643393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08e793-a473-4e64-9404-24c54617743d None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814/groups/28407191aa01492d8ce871e63345bb94/roles/c8ee1507247e4b34a213a8033c46378f` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.650434 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08e793-a473-4e64-9404-24c54617743d None admin] RBAC: Authorizing `identity:create_grant(project_id=66c8b57a8c8e424baf3afade22505814, group_id=28407191aa01492d8ce871e63345bb94, role_id=c8ee1507247e4b34a213a8033c46378f)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:29.651913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08e793-a473-4e64-9404-24c54617743d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:29.675925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 195/416] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:15:29 2022] PUT /identity/v3/projects/66c8b57a8c8e424baf3afade22505814/groups/28407191aa01492d8ce871e63345bb94/roles/c8ee1507247e4b34a213a8033c46378f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:29.679114 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 222/417] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:29.704516 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08e793-a473-4e64-9404-24c54617743d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:29.825630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08e793-a473-4e64-9404-24c54617743d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:29.868945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22e0f3-f797-405d-b7bf-f53dff186c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:29.870365 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e0f3-f797-405d-b7bf-f53dff186c88 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:29.870657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e0f3-f797-405d-b7bf-f53dff186c88 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:29.870934 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22e0f3-f797-405d-b7bf-f53dff186c88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:29.886181 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22e0f3-f797-405d-b7bf-f53dff186c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-de22e0f3-f797-405d-b7bf-f53dff186c88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:29.893989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:29.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:29.895940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 196/418] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:29.903504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2834d-8944-466f-8c85-50308a0ef3cd None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:30.010957 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2834d-8944-466f-8c85-50308a0ef3cd None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:30.056761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:30.057542 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:30.057826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:30.058104 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:30.058862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] RBAC: Adding query filter params (name=member) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:30.059173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:30.060564 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:30.073201 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 223/419] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:30.079139 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22e0f3-f797-405d-b7bf-f53dff186c88 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:30.185445 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22e0f3-f797-405d-b7bf-f53dff186c88 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:30.224560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd694e-e05e-4bcc-9d67-9d501dc0a7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:30.225244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd694e-e05e-4bcc-9d67-9d501dc0a7da None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:30.225534 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd694e-e05e-4bcc-9d67-9d501dc0a7da None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:30.225812 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd694e-e05e-4bcc-9d67-9d501dc0a7da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:30.300199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd694e-e05e-4bcc-9d67-9d501dc0a7da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:30.301715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd694e-e05e-4bcc-9d67-9d501dc0a7da None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:30.402667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 197/420] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:30.408203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:30.546175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a784-e304-4698-9e57-237b0c24ce2b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:30.595140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1601a-8e48-4666-8cd4-a56cd172515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:30.596005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1601a-8e48-4666-8cd4-a56cd172515b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:30.596288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1601a-8e48-4666-8cd4-a56cd172515b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:30.596564 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1601a-8e48-4666-8cd4-a56cd172515b None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:30.597814 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1601a-8e48-4666-8cd4-a56cd172515b None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:30.599246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1601a-8e48-4666-8cd4-a56cd172515b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:30.600929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 224/421] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:30 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:30.606591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd694e-e05e-4bcc-9d67-9d501dc0a7da None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:30.725451 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd694e-e05e-4bcc-9d67-9d501dc0a7da None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:30.768599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced6fbd-b3d2-46b4-88e2-9f89ec7b3e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:30.769494 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6fbd-b3d2-46b4-88e2-9f89ec7b3e2f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:30.769777 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6fbd-b3d2-46b4-88e2-9f89ec7b3e2f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:30.770055 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6fbd-b3d2-46b4-88e2-9f89ec7b3e2f None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:30.773736 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced6fbd-b3d2-46b4-88e2-9f89ec7b3e2f None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:30.775242 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced6fbd-b3d2-46b4-88e2-9f89ec7b3e2f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:30.776888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 198/422] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:30 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:30.782851 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1601a-8e48-4666-8cd4-a56cd172515b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:30.914365 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1601a-8e48-4666-8cd4-a56cd172515b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:30.942692 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc657d17-b39d-4503-bc9f-f1e5542373af None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:30.943033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc657d17-b39d-4503-bc9f-f1e5542373af None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:30.943255 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc657d17-b39d-4503-bc9f-f1e5542373af None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:30.944630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 199/423] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:30.955416 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7906ac5-b0e7-4461-b766-2604bfbe1489 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:30.955607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7906ac5-b0e7-4461-b766-2604bfbe1489 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:30.955869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7906ac5-b0e7-4461-b766-2604bfbe1489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:30.964462 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:30.965151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:30.965431 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:30.965703 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:30.966540 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] RBAC: Adding query filter params (group.id=28407191aa01492d8ce871e63345bb94, role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=84caf63eb1634db7822fe78409be9c77) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:30.966847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:30.968274 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:30.974347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 225/424] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:30 2022] GET /identity/v3/role_assignments?group.id=28407191aa01492d8ce871e63345bb94&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:30.990477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-c7906ac5-b0e7-4461-b766-2604bfbe1489 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:30.991660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-c7906ac5-b0e7-4461-b766-2604bfbe1489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:31.104326 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 200/425] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:31.142134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:31.288354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681f9d9-f5d5-46e4-8a2d-d24eaa40e08e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:31.330191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e7ad8-256a-42b6-b2d4-22d62ecbc960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:31.331000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7ad8-256a-42b6-b2d4-22d62ecbc960 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:31.331278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7ad8-256a-42b6-b2d4-22d62ecbc960 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:31.331545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7ad8-256a-42b6-b2d4-22d62ecbc960 None admin] PATH_INFO: `/v3/roles/member` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:31.342659 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e7ad8-256a-42b6-b2d4-22d62ecbc960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-2e3e7ad8-256a-42b6-b2d4-22d62ecbc960 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:31.350432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:31.351438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:31.352416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 226/426] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:31.359553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7906ac5-b0e7-4461-b766-2604bfbe1489 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:31.466886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6ab08-964a-4ae8-9391-5c35d800ae5f None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:31.467246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6ab08-964a-4ae8-9391-5c35d800ae5f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:31.467447 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6ab08-964a-4ae8-9391-5c35d800ae5f None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:31.469070 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 227/427] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:31.474397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1c80-4237-4401-805d-201db497e6d0 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:31.474668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1c80-4237-4401-805d-201db497e6d0 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:31.474940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1c80-4237-4401-805d-201db497e6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:31.487157 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7906ac5-b0e7-4461-b766-2604bfbe1489 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:31.497638 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-2b8e1c80-4237-4401-805d-201db497e6d0 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:31.498817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-2b8e1c80-4237-4401-805d-201db497e6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:31.522981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:31.523645 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:31.523933 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:31.524244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:31.525133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:31.525449 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:31.526826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:31.535667 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 201/428] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:15:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:31.541107 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:31.580342 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 228/429] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:31.612313 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e1c80-4237-4401-805d-201db497e6d0 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:31.695043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619f1c8-0939-440a-87d7-fa02da59c9f4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:31.733722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e1c80-4237-4401-805d-201db497e6d0 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:31.739255 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ea619-1213-410f-b26e-4ceafdebef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:31.739925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ea619-1213-410f-b26e-4ceafdebef7c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:31.740207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ea619-1213-410f-b26e-4ceafdebef7c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:31.740484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ea619-1213-410f-b26e-4ceafdebef7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:31.771794 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35beff9-a0f0-445a-bd8c-9eef4a1bc50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:31.775398 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35beff9-a0f0-445a-bd8c-9eef4a1bc50e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:31.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35beff9-a0f0-445a-bd8c-9eef4a1bc50e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:31.775944 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35beff9-a0f0-445a-bd8c-9eef4a1bc50e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:31.787461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35beff9-a0f0-445a-bd8c-9eef4a1bc50e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a35beff9-a0f0-445a-bd8c-9eef4a1bc50e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:31.800272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:31.801349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:31.802277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 229/430] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:31.809562 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35beff9-a0f0-445a-bd8c-9eef4a1bc50e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:31.824328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ea619-1213-410f-b26e-4ceafdebef7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:31.825947 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ea619-1213-410f-b26e-4ceafdebef7c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:31.925461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35beff9-a0f0-445a-bd8c-9eef4a1bc50e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:31.935401 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 202/431] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:15:31.936553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ea619-1213-410f-b26e-4ceafdebef7c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:31.966187 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:31.967797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:31.968272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:31.968739 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:31.969712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:31.970345 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:31.972689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:31.984134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 230/432] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:31.989152 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.042233 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ea619-1213-410f-b26e-4ceafdebef7c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.076793 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe436ab-0b9a-426f-b352-27c78f47405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.077733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe436ab-0b9a-426f-b352-27c78f47405f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.078015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe436ab-0b9a-426f-b352-27c78f47405f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.078289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe436ab-0b9a-426f-b352-27c78f47405f None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.079709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe436ab-0b9a-426f-b352-27c78f47405f None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:32.081374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe436ab-0b9a-426f-b352-27c78f47405f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:32.088032 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 203/433] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:31 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:32.089269 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe436ab-0b9a-426f-b352-27c78f47405f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.122243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce3a26-abc2-452d-a5f5-4a8c264d4550 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.165721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039613b-8c3d-4806-b0b2-ef1d2ae627bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.167176 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039613b-8c3d-4806-b0b2-ef1d2ae627bc None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.167740 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039613b-8c3d-4806-b0b2-ef1d2ae627bc None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.168260 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039613b-8c3d-4806-b0b2-ef1d2ae627bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.195705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe436ab-0b9a-426f-b352-27c78f47405f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.233151 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a625368-69c3-49cc-acc8-eab5f2770790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.233988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a625368-69c3-49cc-acc8-eab5f2770790 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.234273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a625368-69c3-49cc-acc8-eab5f2770790 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.234552 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a625368-69c3-49cc-acc8-eab5f2770790 None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.235970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a625368-69c3-49cc-acc8-eab5f2770790 None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:32.237409 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a625368-69c3-49cc-acc8-eab5f2770790 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:32.243974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 204/434] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:32.245105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a625368-69c3-49cc-acc8-eab5f2770790 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.249213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039613b-8c3d-4806-b0b2-ef1d2ae627bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:32.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039613b-8c3d-4806-b0b2-ef1d2ae627bc None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:32.351865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 231/435] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:32.355018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4beced4-7b3c-4b6e-a77b-a070e09d6c41 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.355525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4beced4-7b3c-4b6e-a77b-a070e09d6c41 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.356009 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4beced4-7b3c-4b6e-a77b-a070e09d6c41 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.356879 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a625368-69c3-49cc-acc8-eab5f2770790 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.357781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 232/436] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:32.358806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4beced4-7b3c-4b6e-a77b-a070e09d6c41 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.391094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa34cb1-5f1d-4ddf-a7b3-9c040cbaf91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.391743 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa34cb1-5f1d-4ddf-a7b3-9c040cbaf91a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.392023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa34cb1-5f1d-4ddf-a7b3-9c040cbaf91a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.392295 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa34cb1-5f1d-4ddf-a7b3-9c040cbaf91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.393217 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa34cb1-5f1d-4ddf-a7b3-9c040cbaf91a None admin] RBAC: Adding query filter params (group.id=28407191aa01492d8ce871e63345bb94, role.id=77a3914adfe94a78b2474e4d74d2bf57, scope.project.id=66c8b57a8c8e424baf3afade22505814) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:32.393574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa34cb1-5f1d-4ddf-a7b3-9c040cbaf91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:32.394942 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa34cb1-5f1d-4ddf-a7b3-9c040cbaf91a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:32.401247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 205/437] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/role_assignments?group.id=28407191aa01492d8ce871e63345bb94&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=77a3914adfe94a78b2474e4d74d2bf57 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:32.402519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4871b-7056-44b8-a787-6d7fa3349f90 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.402519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4871b-7056-44b8-a787-6d7fa3349f90 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.402719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4871b-7056-44b8-a787-6d7fa3349f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.420272 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-69a4871b-7056-44b8-a787-6d7fa3349f90 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:32.421543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-69a4871b-7056-44b8-a787-6d7fa3349f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:32.466747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4beced4-7b3c-4b6e-a77b-a070e09d6c41 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.503170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 206/438] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:32.503750 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5dba-b86c-4788-a574-b3274a76748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.504604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64357bfb-b781-4333-a03b-10a36a04135c None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.504899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64357bfb-b781-4333-a03b-10a36a04135c None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.505211 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64357bfb-b781-4333-a03b-10a36a04135c None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.505565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5dba-b86c-4788-a574-b3274a76748a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.506044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5dba-b86c-4788-a574-b3274a76748a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.506525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace5dba-b86c-4788-a574-b3274a76748a None admin] PATH_INFO: `/v3/projects/84caf63eb1634db7822fe78409be9c77` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.510779 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 207/439] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:32.512168 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34955196-a26b-44b3-95d1-800319d5f8b1 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.512579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34955196-a26b-44b3-95d1-800319d5f8b1 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.512751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34955196-a26b-44b3-95d1-800319d5f8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.514197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5dba-b86c-4788-a574-b3274a76748a None admin] RBAC: Authorizing `identity:get_project(project_id=84caf63eb1634db7822fe78409be9c77)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:32.516080 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace5dba-b86c-4788-a574-b3274a76748a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:32.518183 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 233/440] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/projects/84caf63eb1634db7822fe78409be9c77 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:32.523677 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5dba-b86c-4788-a574-b3274a76748a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.531526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-34955196-a26b-44b3-95d1-800319d5f8b1 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:32.532748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-34955196-a26b-44b3-95d1-800319d5f8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:32.604553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 208/441] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:32.605731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34955196-a26b-44b3-95d1-800319d5f8b1 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.634817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace5dba-b86c-4788-a574-b3274a76748a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.670442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca9b6-7633-408b-8144-14298fbefedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.671299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca9b6-7633-408b-8144-14298fbefedd None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.671577 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca9b6-7633-408b-8144-14298fbefedd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.671846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dca9b6-7633-408b-8144-14298fbefedd None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.673012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dca9b6-7633-408b-8144-14298fbefedd None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:32.674531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dca9b6-7633-408b-8144-14298fbefedd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:32.676195 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 234/442] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:32.676941 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca9b6-7633-408b-8144-14298fbefedd None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.706845 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34955196-a26b-44b3-95d1-800319d5f8b1 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.738576 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7103d4-e815-4fb9-adcb-b560aa044e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.739351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7103d4-e815-4fb9-adcb-b560aa044e7b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.739628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7103d4-e815-4fb9-adcb-b560aa044e7b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.739899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7103d4-e815-4fb9-adcb-b560aa044e7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.749747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7103d4-e815-4fb9-adcb-b560aa044e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-cc7103d4-e815-4fb9-adcb-b560aa044e7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:32.757238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:32.758428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 209/443] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:32.760090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7103d4-e815-4fb9-adcb-b560aa044e7b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.777528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dca9b6-7633-408b-8144-14298fbefedd None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.808770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f004de0-8adf-4a28-a6d6-38eec0d6e3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.809681 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f004de0-8adf-4a28-a6d6-38eec0d6e3d0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.809962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f004de0-8adf-4a28-a6d6-38eec0d6e3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.810237 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f004de0-8adf-4a28-a6d6-38eec0d6e3d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.819925 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f004de0-8adf-4a28-a6d6-38eec0d6e3d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-2f004de0-8adf-4a28-a6d6-38eec0d6e3d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:32.828525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:32.829595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:32.830517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 235/444] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:32.831345 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f004de0-8adf-4a28-a6d6-38eec0d6e3d0 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.859152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7103d4-e815-4fb9-adcb-b560aa044e7b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.890832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.891476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.891760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.892036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.892860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] RBAC: Adding query filter params (group.id=28407191aa01492d8ce871e63345bb94, role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=84caf63eb1634db7822fe78409be9c77) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:32.893204 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:32.894643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:32.900746 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 210/445] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/role_assignments?group.id=28407191aa01492d8ce871e63345bb94&scope.project.id=84caf63eb1634db7822fe78409be9c77&role.id=c8ee1507247e4b34a213a8033c46378f => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:32.901540 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:32.935829 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f004de0-8adf-4a28-a6d6-38eec0d6e3d0 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:32.972537 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:32.973250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:32.973530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:32.973799 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:32.974559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:32.974864 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:32.976298 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:32.985194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 236/446] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:32.991203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:33.011815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97947748-5cc1-47be-95ed-90e8f963475c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.047219 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.047901 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.048185 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.048462 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.049264 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:33.049572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.050956 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.059507 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 211/447] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:33.065200 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:33.104093 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522a6d8-f7ba-4a65-a0cb-870283754ac4 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.141635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76b22a-ff5f-4c77-be21-6d300f4ff9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.142247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76b22a-ff5f-4c77-be21-6d300f4ff9f6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.142528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76b22a-ff5f-4c77-be21-6d300f4ff9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.142797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76b22a-ff5f-4c77-be21-6d300f4ff9f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.169558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8939f82-0486-4d5f-931b-8c462735900f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.203767 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abbece-d538-4647-988c-19e6898d4873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.204719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abbece-d538-4647-988c-19e6898d4873 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.205004 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abbece-d538-4647-988c-19e6898d4873 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.205320 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abbece-d538-4647-988c-19e6898d4873 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.219757 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76b22a-ff5f-4c77-be21-6d300f4ff9f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.221493 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76b22a-ff5f-4c77-be21-6d300f4ff9f6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.275504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abbece-d538-4647-988c-19e6898d4873 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.276965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abbece-d538-4647-988c-19e6898d4873 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.335722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 237/448] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:15:33.342111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76b22a-ff5f-4c77-be21-6d300f4ff9f6 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:33.383797 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 212/449] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:33.389214 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abbece-d538-4647-988c-19e6898d4873 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:33.450795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76b22a-ff5f-4c77-be21-6d300f4ff9f6 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.482614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ec74f-6de7-4aac-81d1-f7bb0412d0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.483465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ec74f-6de7-4aac-81d1-f7bb0412d0c1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.483742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ec74f-6de7-4aac-81d1-f7bb0412d0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.484007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ec74f-6de7-4aac-81d1-f7bb0412d0c1 None admin] PATH_INFO: `/v3/projects/2f91da8032e14759941fd956b26044bc` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.485206 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ec74f-6de7-4aac-81d1-f7bb0412d0c1 None admin] RBAC: Authorizing `identity:get_project(project_id=2f91da8032e14759941fd956b26044bc)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.486637 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ec74f-6de7-4aac-81d1-f7bb0412d0c1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.488079 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abbece-d538-4647-988c-19e6898d4873 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.488308 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 238/450] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:33 2022] GET /identity/v3/projects/2f91da8032e14759941fd956b26044bc => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:33.494702 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ec74f-6de7-4aac-81d1-f7bb0412d0c1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:33.521172 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc49b3-1b43-4327-afc7-625cd033247a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.522021 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc49b3-1b43-4327-afc7-625cd033247a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.522300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc49b3-1b43-4327-afc7-625cd033247a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.522573 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc49b3-1b43-4327-afc7-625cd033247a None admin] PATH_INFO: `/v3/projects/66c8b57a8c8e424baf3afade22505814` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.523757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc49b3-1b43-4327-afc7-625cd033247a None admin] RBAC: Authorizing `identity:get_project(project_id=66c8b57a8c8e424baf3afade22505814)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.525231 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc49b3-1b43-4327-afc7-625cd033247a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.526904 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 213/451] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:15:33 2022] GET /identity/v3/projects/66c8b57a8c8e424baf3afade22505814 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:33.532129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc49b3-1b43-4327-afc7-625cd033247a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:33.586148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ec74f-6de7-4aac-81d1-f7bb0412d0c1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.617769 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b80da0-c334-4eda-9164-93e2d003dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.619413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80da0-c334-4eda-9164-93e2d003dd71 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.619689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80da0-c334-4eda-9164-93e2d003dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.619956 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80da0-c334-4eda-9164-93e2d003dd71 None admin] PATH_INFO: `/v3/groups/7c613ef05cbd4156a8010186ea679681` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.621174 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b80da0-c334-4eda-9164-93e2d003dd71 None admin] RBAC: Authorizing `identity:get_group(group_id=7c613ef05cbd4156a8010186ea679681)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.622601 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b80da0-c334-4eda-9164-93e2d003dd71 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.623969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc49b3-1b43-4327-afc7-625cd033247a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.624236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 239/452] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:33 2022] GET /identity/v3/groups/7c613ef05cbd4156a8010186ea679681 => generated 229 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:33.629855 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b80da0-c334-4eda-9164-93e2d003dd71 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:33.655203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e52860-e768-4a69-9085-3810bf2528b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.656038 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e52860-e768-4a69-9085-3810bf2528b3 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.656313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e52860-e768-4a69-9085-3810bf2528b3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.656579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e52860-e768-4a69-9085-3810bf2528b3 None admin] PATH_INFO: `/v3/groups/28407191aa01492d8ce871e63345bb94` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.657767 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e52860-e768-4a69-9085-3810bf2528b3 None admin] RBAC: Authorizing `identity:get_group(group_id=28407191aa01492d8ce871e63345bb94)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.659173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e52860-e768-4a69-9085-3810bf2528b3 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.660754 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 214/453] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:15:33 2022] GET /identity/v3/groups/28407191aa01492d8ce871e63345bb94 => generated 226 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:33.666838 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e52860-e768-4a69-9085-3810bf2528b3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:33.721923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b80da0-c334-4eda-9164-93e2d003dd71 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.756062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f8fb4-04a2-4870-9e78-51ccfcfa1329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.756710 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8fb4-04a2-4870-9e78-51ccfcfa1329 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.757002 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8fb4-04a2-4870-9e78-51ccfcfa1329 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.757300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f8fb4-04a2-4870-9e78-51ccfcfa1329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.758123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f8fb4-04a2-4870-9e78-51ccfcfa1329 None admin] RBAC: Adding query filter params (group.id=7c613ef05cbd4156a8010186ea679681, role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=2f91da8032e14759941fd956b26044bc) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:33.758453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f8fb4-04a2-4870-9e78-51ccfcfa1329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.759101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e52860-e768-4a69-9085-3810bf2528b3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:33.759878 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f8fb4-04a2-4870-9e78-51ccfcfa1329 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.765682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 240/454] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:33 2022] GET /identity/v3/role_assignments?group.id=7c613ef05cbd4156a8010186ea679681&scope.project.id=2f91da8032e14759941fd956b26044bc&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:33.793968 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c7d9a-e304-475f-9c20-4a8c0eebd51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:33.794641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7d9a-e304-475f-9c20-4a8c0eebd51d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:33.794922 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7d9a-e304-475f-9c20-4a8c0eebd51d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:33.795195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7d9a-e304-475f-9c20-4a8c0eebd51d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:33.796045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c7d9a-e304-475f-9c20-4a8c0eebd51d None admin] RBAC: Adding query filter params (group.id=28407191aa01492d8ce871e63345bb94, role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=66c8b57a8c8e424baf3afade22505814) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:33.796356 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c7d9a-e304-475f-9c20-4a8c0eebd51d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:33.797879 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c7d9a-e304-475f-9c20-4a8c0eebd51d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:33.804414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 215/455] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:15:33 2022] GET /identity/v3/role_assignments?group.id=28407191aa01492d8ce871e63345bb94&scope.project.id=66c8b57a8c8e424baf3afade22505814&role.id=c8ee1507247e4b34a213a8033c46378f => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:36.731232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca06a0-b726-42ea-a892-ab098c843f43 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:36.732259 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca06a0-b726-42ea-a892-ab098c843f43 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:36.732791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca06a0-b726-42ea-a892-ab098c843f43 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:36.734666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 241/456] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:36.740444 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e017a5-9e22-47eb-aa6a-1accc1d91740 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:36.741272 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e017a5-9e22-47eb-aa6a-1accc1d91740 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:36.741734 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e017a5-9e22-47eb-aa6a-1accc1d91740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:36.763062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-00e017a5-9e22-47eb-aa6a-1accc1d91740 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:36.764102 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f161d12-80dd-42bd-a33a-ca04e8dab71c None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:36.764560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-00e017a5-9e22-47eb-aa6a-1accc1d91740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:36.764871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f161d12-80dd-42bd-a33a-ca04e8dab71c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:36.765402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f161d12-80dd-42bd-a33a-ca04e8dab71c None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:36.767273 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 242/457] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:36.772836 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c01b18-6570-430f-aac4-c89519a954a4 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:36.773445 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c01b18-6570-430f-aac4-c89519a954a4 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:36.773933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c01b18-6570-430f-aac4-c89519a954a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:36.791558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-e0c01b18-6570-430f-aac4-c89519a954a4 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:36.793172 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-e0c01b18-6570-430f-aac4-c89519a954a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:36.855469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 216/458] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:36.872612 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 243/459] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:36.901491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e017a5-9e22-47eb-aa6a-1accc1d91740 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:36.933769 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c01b18-6570-430f-aac4-c89519a954a4 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:37.037026 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c01b18-6570-430f-aac4-c89519a954a4 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:37.060634 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e017a5-9e22-47eb-aa6a-1accc1d91740 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:37.072568 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:37.073713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.074243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.074744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.085205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:37.086946 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:37.105492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:37.106482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:37.107435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:37.107435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:37.107435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:37.107435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:37.109790 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 244/460] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:37.115728 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:37.144665 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:37.145939 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.146462 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.146955 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.171112 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:37.172589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:37.211101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:37.212175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:37.213173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:37.213173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:37.213173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:37.213173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:37.213173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 217/461] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:37.225914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:37.253891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45a34e-0ecb-4737-9c10-381dd465ba43 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:37.300227 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a4478-5e9c-493d-933d-23fb12ee5e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:37.301361 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a4478-5e9c-493d-933d-23fb12ee5e15 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.302896 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a4478-5e9c-493d-933d-23fb12ee5e15 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.303415 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a4478-5e9c-493d-933d-23fb12ee5e15 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.304696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a4478-5e9c-493d-933d-23fb12ee5e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:37.307366 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a4478-5e9c-493d-933d-23fb12ee5e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:37.309123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a4478-5e9c-493d-933d-23fb12ee5e15 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:37.323388 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 245/462] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:37.325291 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cb21f-2054-402e-8821-64f068030ada None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.326732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cb21f-2054-402e-8821-64f068030ada None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.327246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cb21f-2054-402e-8821-64f068030ada None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.329150 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 246/463] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:37.330092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cb21f-2054-402e-8821-64f068030ada None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:37.400538 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ab736-3e26-4823-b786-af29eb23075d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:37.436052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb3fad-e49c-4a6d-b385-b3ef2ce6db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:37.437162 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3fad-e49c-4a6d-b385-b3ef2ce6db9e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.437610 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3fad-e49c-4a6d-b385-b3ef2ce6db9e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.438046 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb3fad-e49c-4a6d-b385-b3ef2ce6db9e None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.438912 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb3fad-e49c-4a6d-b385-b3ef2ce6db9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:37.439378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb3fad-e49c-4a6d-b385-b3ef2ce6db9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:37.440899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb3fad-e49c-4a6d-b385-b3ef2ce6db9e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:37.460278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 218/464] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:37.461903 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cb21f-2054-402e-8821-64f068030ada None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:37.462266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66d7c5-5389-4628-9918-f847e72dfe13 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.462702 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66d7c5-5389-4628-9918-f847e72dfe13 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.463148 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66d7c5-5389-4628-9918-f847e72dfe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.479636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-1e66d7c5-5389-4628-9918-f847e72dfe13 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:37.481271 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-1e66d7c5-5389-4628-9918-f847e72dfe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:37.505151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253022af-d1d9-4eb3-8b20-f11b6fce8c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:37.506608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253022af-d1d9-4eb3-8b20-f11b6fce8c43 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.507059 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253022af-d1d9-4eb3-8b20-f11b6fce8c43 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.507753 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253022af-d1d9-4eb3-8b20-f11b6fce8c43 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.509364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253022af-d1d9-4eb3-8b20-f11b6fce8c43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:37.511580 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253022af-d1d9-4eb3-8b20-f11b6fce8c43 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:37.541348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 247/465] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:37 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:15:37.541348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253022af-d1d9-4eb3-8b20-f11b6fce8c43 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:37.562243 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 219/466] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:37.563712 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6821b8-17fd-4a23-ac52-7a2c302793a2 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.564287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6821b8-17fd-4a23-ac52-7a2c302793a2 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.564772 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6821b8-17fd-4a23-ac52-7a2c302793a2 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.566375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 220/467] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:37.573171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d1ae7-8e07-4bf6-a5f8-01a45a0e96e5 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.573663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d1ae7-8e07-4bf6-a5f8-01a45a0e96e5 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.574090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d1ae7-8e07-4bf6-a5f8-01a45a0e96e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.595309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-284d1ae7-8e07-4bf6-a5f8-01a45a0e96e5 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:37.597324 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-284d1ae7-8e07-4bf6-a5f8-01a45a0e96e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:37.677862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253022af-d1d9-4eb3-8b20-f11b6fce8c43 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:37.679035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 221/468] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:37.680773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d1ae7-8e07-4bf6-a5f8-01a45a0e96e5 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:37.717828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354553d-592c-4eee-8184-b25702e3944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:37.718931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354553d-592c-4eee-8184-b25702e3944d None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.719417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354553d-592c-4eee-8184-b25702e3944d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.719888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354553d-592c-4eee-8184-b25702e3944d None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.720826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354553d-592c-4eee-8184-b25702e3944d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:37.722546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354553d-592c-4eee-8184-b25702e3944d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:37.751695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 248/469] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:37 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:15:37.753453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354553d-592c-4eee-8184-b25702e3944d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:37.836513 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d1ae7-8e07-4bf6-a5f8-01a45a0e96e5 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:37.882735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:37.884194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.884719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.885232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.885663 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354553d-592c-4eee-8184-b25702e3944d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:37.896574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:37.898210 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:37.919374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:37.920417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:37.921427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:37.921427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:37.921427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:37.921427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:37.921427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 222/470] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:37.928431 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:37.939398 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:37.940636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:37.941756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:37.942262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:37.956543 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:37.959145 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:37.982041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:37.983064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:37.984023 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:37.984023 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:37.984023 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:37.984023 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:37.986677 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 249/471] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:37.992514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:38.064393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02912619-50f3-4f5f-87ca-f49e540eae76 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:38.103602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014df955-72a2-4034-8a79-da7b16430624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:38.104317 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df955-72a2-4034-8a79-da7b16430624 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.104664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df955-72a2-4034-8a79-da7b16430624 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.105020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014df955-72a2-4034-8a79-da7b16430624 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.105793 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014df955-72a2-4034-8a79-da7b16430624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:38.106108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014df955-72a2-4034-8a79-da7b16430624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:38.107223 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb40905-9798-41f0-b75a-99835d058a46 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:38.107496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014df955-72a2-4034-8a79-da7b16430624 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:38.121433 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 223/472] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:38.122826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cad8c-183b-4bed-a896-110d65bec220 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.123171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cad8c-183b-4bed-a896-110d65bec220 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.123392 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cad8c-183b-4bed-a896-110d65bec220 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.124639 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 224/473] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:38.127657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cad8c-183b-4bed-a896-110d65bec220 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:38.160424 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85764c0c-9763-44c8-8940-bc429ddc26ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:38.161352 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85764c0c-9763-44c8-8940-bc429ddc26ec None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.161821 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85764c0c-9763-44c8-8940-bc429ddc26ec None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.162164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85764c0c-9763-44c8-8940-bc429ddc26ec None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.162971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85764c0c-9763-44c8-8940-bc429ddc26ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:38.163352 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85764c0c-9763-44c8-8940-bc429ddc26ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:38.166120 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85764c0c-9763-44c8-8940-bc429ddc26ec None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:38.184158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 250/474] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:38.184158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e5f9-0608-4b85-bf55-0798a7b96ca6 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.184158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e5f9-0608-4b85-bf55-0798a7b96ca6 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.184158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e5f9-0608-4b85-bf55-0798a7b96ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.203839 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-5dc0e5f9-0608-4b85-bf55-0798a7b96ca6 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:38.205443 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-5dc0e5f9-0608-4b85-bf55-0798a7b96ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:38.248507 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cad8c-183b-4bed-a896-110d65bec220 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:38.292741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 251/475] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:38.293677 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db67ff-791a-4dc0-ac36-cbd9778131a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:38.294513 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db67ff-791a-4dc0-ac36-cbd9778131a7 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.295004 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db67ff-791a-4dc0-ac36-cbd9778131a7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.295440 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db67ff-791a-4dc0-ac36-cbd9778131a7 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.296292 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db67ff-791a-4dc0-ac36-cbd9778131a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:38.297906 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db67ff-791a-4dc0-ac36-cbd9778131a7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:38.298909 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e5f9-0608-4b85-bf55-0798a7b96ca6 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:38.332923 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 225/476] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:38 2022] POST /identity/v3/users => generated 249 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:15:38.334351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db67ff-791a-4dc0-ac36-cbd9778131a7 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:38.492490 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e5f9-0608-4b85-bf55-0798a7b96ca6 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:38.512271 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db67ff-791a-4dc0-ac36-cbd9778131a7 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:38.532135 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb947ed1-618c-4b1f-889e-1dc722b3ef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:38.533233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb947ed1-618c-4b1f-889e-1dc722b3ef94 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.533840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb947ed1-618c-4b1f-889e-1dc722b3ef94 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.534358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb947ed1-618c-4b1f-889e-1dc722b3ef94 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.535281 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb947ed1-618c-4b1f-889e-1dc722b3ef94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:38.536961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb947ed1-618c-4b1f-889e-1dc722b3ef94 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:38.552907 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:38.553653 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.553902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.554178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.554924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:38.556313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:38.560130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 252/477] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:38 2022] POST /identity/v3/users => generated 248 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:15:38.564948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.sql.core [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:15:38.564948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 15:15:38.564948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [parameters: {'id': '73e554e6edb04e1db80ed1d11994b3e3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 15:15:38.564948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=143116) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application conn.query(q) May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application result.read() May 09 15:15:38.582842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return ref.to_dict() May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:15:38.583975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application next(self.gen) May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application yield resource May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application next(self.gen) May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise self.value May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application session.commit() May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:15:38.585063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.session.flush() May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application compat.raise_( May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:15:38.586059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:15:38.587889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application conn.query(q) May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application result.read() May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:15:38.589014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [parameters: {'id': '73e554e6edb04e1db80ed1d11994b3e3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:38.590052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 15:15:38.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:38.591932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 226/478] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:15:38 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:38.593379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb947ed1-618c-4b1f-889e-1dc722b3ef94 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:38.715374 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb947ed1-618c-4b1f-889e-1dc722b3ef94 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:38.757526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde40d54-fb52-4712-84a8-d3f2fae0815d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:38.758833 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40d54-fb52-4712-84a8-d3f2fae0815d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.759112 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40d54-fb52-4712-84a8-d3f2fae0815d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.759385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde40d54-fb52-4712-84a8-d3f2fae0815d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.777458 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde40d54-fb52-4712-84a8-d3f2fae0815d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-fde40d54-fb52-4712-84a8-d3f2fae0815d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:38.786976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:38.788015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:38.788929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 253/479] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:15:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:38.794420 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:38.909377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46305aa-c621-417b-9e80-67e706c41733 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:38.943953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6b03a-2557-4c2a-8674-8d6062b8df77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:38.944761 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b03a-2557-4c2a-8674-8d6062b8df77 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:38.945182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b03a-2557-4c2a-8674-8d6062b8df77 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:38.945559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b03a-2557-4c2a-8674-8d6062b8df77 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:38.946982 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b03a-2557-4c2a-8674-8d6062b8df77 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:38.947391 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b03a-2557-4c2a-8674-8d6062b8df77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:38.948879 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b03a-2557-4c2a-8674-8d6062b8df77 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:38.957415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 227/480] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:40.379789 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d18c3-316c-4fb9-96b6-acca1b436e45 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:40.380353 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d18c3-316c-4fb9-96b6-acca1b436e45 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:40.380353 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d18c3-316c-4fb9-96b6-acca1b436e45 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:40.382051 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 254/481] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:40.388179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c205e14-87f5-4fa7-8a28-27ce7e51c770 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:40.388669 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c205e14-87f5-4fa7-8a28-27ce7e51c770 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:40.388857 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c205e14-87f5-4fa7-8a28-27ce7e51c770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:40.414043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-0c205e14-87f5-4fa7-8a28-27ce7e51c770 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:40.415283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-0c205e14-87f5-4fa7-8a28-27ce7e51c770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:40.487277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 228/482] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:40.529782 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d18c3-316c-4fb9-96b6-acca1b436e45 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:40.653370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991826b0-3d24-46d2-8293-e9679abc5d9c None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:40.653708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991826b0-3d24-46d2-8293-e9679abc5d9c None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:40.653932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991826b0-3d24-46d2-8293-e9679abc5d9c None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:40.655593 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 229/483] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:15:40.656475 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d18c3-316c-4fb9-96b6-acca1b436e45 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:40.671176 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b4a40-3b9e-4ce1-9a07-3b808d0cf9ab None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:40.671759 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b4a40-3b9e-4ce1-9a07-3b808d0cf9ab None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:40.672267 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b4a40-3b9e-4ce1-9a07-3b808d0cf9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:40.703325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-a85b4a40-3b9e-4ce1-9a07-3b808d0cf9ab None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:40.704869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-a85b4a40-3b9e-4ce1-9a07-3b808d0cf9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:40.706306 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e185ecf-ecf2-43df-be3f-40371467eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:40.707127 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e185ecf-ecf2-43df-be3f-40371467eee1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:40.707411 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e185ecf-ecf2-43df-be3f-40371467eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:40.707690 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e185ecf-ecf2-43df-be3f-40371467eee1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:40.726149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e185ecf-ecf2-43df-be3f-40371467eee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-5e185ecf-ecf2-43df-be3f-40371467eee1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:40.743453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:40.745101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:40.746263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 255/484] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:40.749819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e185ecf-ecf2-43df-be3f-40371467eee1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:40.779948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 230/485] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:40.826863 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b4a40-3b9e-4ce1-9a07-3b808d0cf9ab None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:40.852531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e185ecf-ecf2-43df-be3f-40371467eee1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:40.888551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:40.889280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:40.889532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:40.889810 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:40.890581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:40.890890 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:40.892287 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:40.912549 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 256/486] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:40.917813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:40.968227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b4a40-3b9e-4ce1-9a07-3b808d0cf9ab None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.006206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e8119-4707-44c1-82e2-1bc0d462b3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.007441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e8119-4707-44c1-82e2-1bc0d462b3ed None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.007966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e8119-4707-44c1-82e2-1bc0d462b3ed None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.008981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e8119-4707-44c1-82e2-1bc0d462b3ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.021031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e8119-4707-44c1-82e2-1bc0d462b3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-849e8119-4707-44c1-82e2-1bc0d462b3ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:41.030807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:41.031829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:41.032842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:41.036311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 231/487] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:41.042768 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e8119-4707-44c1-82e2-1bc0d462b3ed None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.163528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad64e43-01d9-4299-8867-ba28f1ceddd9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.173448 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e8119-4707-44c1-82e2-1bc0d462b3ed None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.199456 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c94c35-bfe4-48fc-8fc5-932982134149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.200411 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c94c35-bfe4-48fc-8fc5-932982134149 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.200554 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c94c35-bfe4-48fc-8fc5-932982134149 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.200783 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c94c35-bfe4-48fc-8fc5-932982134149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.211512 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c94c35-bfe4-48fc-8fc5-932982134149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:41.212823 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c94c35-bfe4-48fc-8fc5-932982134149 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:41.217096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.218078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.218573 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.219445 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.220816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] RBAC: Adding query filter params (name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:41.221636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:41.223294 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-16c94c35-bfe4-48fc-8fc5-932982134149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:41.229451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:41.230432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:41.232004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:41.232004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:41.232004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:41.232004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:41.232004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 257/488] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:41.232323 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf5e1a-db07-4738-99e0-8b27d1c238f2 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.232555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf5e1a-db07-4738-99e0-8b27d1c238f2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.232744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf5e1a-db07-4738-99e0-8b27d1c238f2 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.235886 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 232/490] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:41.237001 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.241537 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 258/490] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:41.242141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf5e1a-db07-4738-99e0-8b27d1c238f2 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.352186 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf5e1a-db07-4738-99e0-8b27d1c238f2 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.371048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2c7a-5e01-4011-8308-7007676ab6cf None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.406805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.407717 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.408082 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.408403 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.410602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b51bb-8157-4548-a398-1d3bcb5f34f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.411324 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b51bb-8157-4548-a398-1d3bcb5f34f1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.411672 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b51bb-8157-4548-a398-1d3bcb5f34f1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.412029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b51bb-8157-4548-a398-1d3bcb5f34f1 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.412828 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b51bb-8157-4548-a398-1d3bcb5f34f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:41.413239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b51bb-8157-4548-a398-1d3bcb5f34f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:41.414743 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b51bb-8157-4548-a398-1d3bcb5f34f1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:41.422828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:41.425365 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:41.427394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 233/491] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:41.428688 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb499ea6-19b0-42a1-a0f8-5e0e4de8847f None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.429020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb499ea6-19b0-42a1-a0f8-5e0e4de8847f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.429398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb499ea6-19b0-42a1-a0f8-5e0e4de8847f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.447278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-eb499ea6-19b0-42a1-a0f8-5e0e4de8847f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:41.447732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:41.448687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:41.449779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:41.449779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:41.449779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:41.449779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:41.450027 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-eb499ea6-19b0-42a1-a0f8-5e0e4de8847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:41.452854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 259/492] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:41.453993 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.529146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 234/493] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:41.530109 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb499ea6-19b0-42a1-a0f8-5e0e4de8847f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.571506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a795db-dcd3-438e-8ded-9ed0225f53df None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.604211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e116c60-efbc-44fe-aa84-fb37addfce35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.604924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e116c60-efbc-44fe-aa84-fb37addfce35 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.605223 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e116c60-efbc-44fe-aa84-fb37addfce35 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.605490 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e116c60-efbc-44fe-aa84-fb37addfce35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.612924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e116c60-efbc-44fe-aa84-fb37addfce35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:41.614413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e116c60-efbc-44fe-aa84-fb37addfce35 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-3e116c60-efbc-44fe-aa84-fb37addfce35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:41.630550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:41.631832 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:41.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 260/494] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:41.633741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da626e-e21a-4dfe-af9c-b2e675caad0a None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.633741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da626e-e21a-4dfe-af9c-b2e675caad0a None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.633741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da626e-e21a-4dfe-af9c-b2e675caad0a None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.638922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 261/495] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:41.638922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da626e-e21a-4dfe-af9c-b2e675caad0a None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.647464 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb499ea6-19b0-42a1-a0f8-5e0e4de8847f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.682170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.682834 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.683128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.683411 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.684141 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:41.684450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:41.685891 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:41.696811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 235/496] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:41.697641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.750616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da626e-e21a-4dfe-af9c-b2e675caad0a None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.786899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d43d5-4013-432a-8db9-457d17b83018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.787695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d43d5-4013-432a-8db9-457d17b83018 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.787974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d43d5-4013-432a-8db9-457d17b83018 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.788246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d43d5-4013-432a-8db9-457d17b83018 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.799927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d43d5-4013-432a-8db9-457d17b83018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:41.806341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3b70-d069-4faf-9f3a-18c66c5b7b54 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-785d43d5-4013-432a-8db9-457d17b83018 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:41.808907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:41.809992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 262/497] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:41.815955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c7813-0517-4780-808a-1e0e9bac8c8d None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.816815 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c7813-0517-4780-808a-1e0e9bac8c8d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.816815 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c7813-0517-4780-808a-1e0e9bac8c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.831072 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-1f3c7813-0517-4780-808a-1e0e9bac8c8d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:41.832279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-1f3c7813-0517-4780-808a-1e0e9bac8c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:41.840164 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:41.840825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:41.841134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:41.841412 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:41.842239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:41.842541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:41.843974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:41.884965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 236/498] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:41.884965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.913062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 263/499] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:41.913062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c7813-0517-4780-808a-1e0e9bac8c8d None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:41.993340 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284744c-82bf-4bb6-9691-2a02e8927720 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:41.999465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c7813-0517-4780-808a-1e0e9bac8c8d None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.025890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.026690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.026970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.027247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] PATH_INFO: `/v3/users/glance` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.031302 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.031955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.032238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.032507 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.033273 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:42.033576 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.034129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.035048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.035575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.043121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 264/500] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:42.043896 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:42.052003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:42.053229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 237/501] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:42.054259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.143124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e97fc-15c8-4c2e-a656-54c46d0aa536 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.152421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e0a8-d0d7-442e-8d42-93ec81d8ca3e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.175483 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1774b0-7a75-4610-9838-e61790df5ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.176125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1774b0-7a75-4610-9838-e61790df5ad2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.176408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1774b0-7a75-4610-9838-e61790df5ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.176682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1774b0-7a75-4610-9838-e61790df5ad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.184395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfea012-5fd2-4d77-ad81-14882170dbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.185217 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfea012-5fd2-4d77-ad81-14882170dbfb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.185495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfea012-5fd2-4d77-ad81-14882170dbfb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.185764 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfea012-5fd2-4d77-ad81-14882170dbfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.195447 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfea012-5fd2-4d77-ad81-14882170dbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-7bfea012-5fd2-4d77-ad81-14882170dbfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:42.203133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:42.204378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:42.205370 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 238/502] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:42.206118 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587895b-ddfd-4908-9521-5f6b608448f1 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.206118 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587895b-ddfd-4908-9521-5f6b608448f1 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.206118 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e587895b-ddfd-4908-9521-5f6b608448f1 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.207381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 239/503] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:42.207929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587895b-ddfd-4908-9521-5f6b608448f1 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.245498 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1774b0-7a75-4610-9838-e61790df5ad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.246958 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1774b0-7a75-4610-9838-e61790df5ad2 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.307124 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e587895b-ddfd-4908-9521-5f6b608448f1 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.338700 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.339478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.339759 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.340033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.340348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 265/504] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:42.341225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1774b0-7a75-4610-9838-e61790df5ad2 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.350242 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.351678 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:42.368199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:42.369470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:42.370428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:42.370428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:42.370428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:42.370428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:42.370428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 240/505] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:42.370810 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.438136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1774b0-7a75-4610-9838-e61790df5ad2 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.471321 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce495d0e-de30-4a82-ac4b-2e9550a2fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.471986 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce495d0e-de30-4a82-ac4b-2e9550a2fc20 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.472268 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce495d0e-de30-4a82-ac4b-2e9550a2fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.472544 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce495d0e-de30-4a82-ac4b-2e9550a2fc20 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.473413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce495d0e-de30-4a82-ac4b-2e9550a2fc20 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:42.473723 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce495d0e-de30-4a82-ac4b-2e9550a2fc20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.475185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce495d0e-de30-4a82-ac4b-2e9550a2fc20 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.480879 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76d78f-be15-459a-a673-cc8f2b14128f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.511042 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 266/506] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:42.512497 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ea53-f919-4294-a038-d4b06fe12b89 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.512861 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ea53-f919-4294-a038-d4b06fe12b89 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.513250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ea53-f919-4294-a038-d4b06fe12b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.517483 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.518146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.518429 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.518707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.519455 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:42.519763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.521160 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.529803 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 241/507] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:42.530442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.531500 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-8927ea53-f919-4294-a038-d4b06fe12b89 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:42.532709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-8927ea53-f919-4294-a038-d4b06fe12b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:42.594962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 267/508] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:42.595795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927ea53-f919-4294-a038-d4b06fe12b89 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.624813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b0ecb-825b-4398-9157-3e629ac37fa5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.657206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.657983 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.658263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.658543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.669377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.670817 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:42.687596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:42.688969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:42.690043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:42.690043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:42.690043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:42.690043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:42.690043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 242/509] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:42.690043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.694716 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927ea53-f919-4294-a038-d4b06fe12b89 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.727450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1011116-c298-4049-b154-86437facb546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.728109 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011116-c298-4049-b154-86437facb546 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.728396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011116-c298-4049-b154-86437facb546 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.728674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1011116-c298-4049-b154-86437facb546 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.729437 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1011116-c298-4049-b154-86437facb546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:42.729749 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1011116-c298-4049-b154-86437facb546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.731132 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1011116-c298-4049-b154-86437facb546 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.741787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 268/510] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:42.743093 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1011116-c298-4049-b154-86437facb546 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.784404 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70202493-34d2-4188-98e5-17d30fcfdff8 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.816192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93078a2-fa01-4472-a91e-eb72a5366997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.816832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93078a2-fa01-4472-a91e-eb72a5366997 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.817147 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93078a2-fa01-4472-a91e-eb72a5366997 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.817423 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93078a2-fa01-4472-a91e-eb72a5366997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.835257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1011116-c298-4049-b154-86437facb546 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:42.866526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:42.867304 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:42.868124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:42.868399 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:42.878720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.880217 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.881143 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93078a2-fa01-4472-a91e-eb72a5366997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:42.882601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93078a2-fa01-4472-a91e-eb72a5366997 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:42.897296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:42.898286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:42.899286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:42.899286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:42.899286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:42.899286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:42.899286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 269/511] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:42.899286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.978666 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 243/512] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:42.979427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93078a2-fa01-4472-a91e-eb72a5366997 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:42.991830 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe7d73-b4d2-4518-8015-1745f7f7147b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.024940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.025754 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.026037 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.026316 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.037737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.039248 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:43.057161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:43.058279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:43.059448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:43.059448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:43.059448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:43.059448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:43.059448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 270/513] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:43.059805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.075195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93078a2-fa01-4472-a91e-eb72a5366997 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.107277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.107934 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.108214 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.108492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.109236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:43.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.110898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.122496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 244/514] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:43.123472 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.157963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cbc11-646f-4647-9b07-14ce0f96a050 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.189024 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.189833 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.190111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.190383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.198074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.199505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:43.213747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:43.214807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 271/515] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:43.215922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.221659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afa23d-a93f-4905-8adf-d5b3425c80be None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.255285 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.256265 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.256265 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.256511 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.257223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:43.257543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.259472 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.270973 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 245/516] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:43.271760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.309898 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f90f1-f6f4-4a6a-82e0-94272bb40944 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.341919 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.342709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.342990 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.343264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.353301 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.354746 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.364463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5484906-4489-45f3-b834-efca682aeeb8 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:43.371326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:43.372344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:43.373478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:43.373478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:43.373478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:43.373478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:43.373478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 272/517] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:43.373786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.399492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.400152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.400438 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.400713 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.401471 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:43.401779 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.403165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.414092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 246/518] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:43.414827 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.468282 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c2990-ef71-413a-8268-e43ce49cb2b5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.500173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.500955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.501265 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.501541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.508079 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.509002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27e587-49b8-47b0-b901-cb6cf8a4a71c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.510153 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:43.524744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:43.525974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 273/519] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:43.526945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.541067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.541729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.542015 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.542302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.543140 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:43.543453 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.544881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.561823 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 247/520] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:43.562600 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.623498 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1f88a-f23a-4a90-887d-03300ae2b431 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.655751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec32d8f-2b4e-4d9e-8e73-85852d05b629 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.655949 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.656617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.656905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.657218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.664074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.666162 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:43.681126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:43.682299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:15:43.683527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:43.685341 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 274/521] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:43.686045 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.688435 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.689129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.689415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.689693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.690415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:43.690722 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.692091 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.702990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 248/522] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:43.703765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.780038 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c026f-6885-4195-8f67-4a57dba15dcd None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.798494 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05215b-2ce8-4c1a-a397-2927a5451da6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.812022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95186c-99ab-4af3-95ef-30b5072ae12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.812684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95186c-99ab-4af3-95ef-30b5072ae12c None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.812965 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95186c-99ab-4af3-95ef-30b5072ae12c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.813272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95186c-99ab-4af3-95ef-30b5072ae12c None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.813977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95186c-99ab-4af3-95ef-30b5072ae12c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.815415 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95186c-99ab-4af3-95ef-30b5072ae12c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.830542 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.831284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.831570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.831918 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:43.832827 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:43.833690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:43.834627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:43.836181 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 275/523] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:15:43 2022] POST /identity/v3/users => generated 247 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:15:43.837617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95186c-99ab-4af3-95ef-30b5072ae12c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.858906 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 249/524] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:43.858906 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:43.947334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95186c-99ab-4af3-95ef-30b5072ae12c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.964724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf7998-4a55-4fd8-859c-8c53bc888fbf None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:43.982264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf20a8e-ff85-4576-879a-834b1bebff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:43.982932 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf20a8e-ff85-4576-879a-834b1bebff53 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:43.983216 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf20a8e-ff85-4576-879a-834b1bebff53 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:43.983490 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf20a8e-ff85-4576-879a-834b1bebff53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.000090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.000779 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.001084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.001363 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.002212 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:44.002520 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.003987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.023204 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 250/525] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:44.023537 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.055661 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf20a8e-ff85-4576-879a-834b1bebff53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.057084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf20a8e-ff85-4576-879a-834b1bebff53 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.132281 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4163e8-53fe-4c1f-9964-0d6a90a8c75d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:44.159873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 276/526] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:44.159873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf20a8e-ff85-4576-879a-834b1bebff53 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.171670 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.172477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.172761 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.173060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.179821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.181893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:44.196708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:44.197873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:44.198859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:44.198859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:44.198859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:44.198859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:44.198859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:44.198859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:44.198859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:44.198859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:44.201354 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 251/527] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:44.202400 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.254262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf20a8e-ff85-4576-879a-834b1bebff53 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:44.290180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.290830 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.291104 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.291414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.292414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=f3a73bc43c23400d8ee7d4e880aa7242) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:44.292529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.293938 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.299354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 277/528] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/role_assignments?user.id=f3a73bc43c23400d8ee7d4e880aa7242&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:44.299956 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.312844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890afec-0fd5-4932-ad15-89ffc7b4a080 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:44.351736 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8aac3f-fa7c-4d5a-8a0b-f66fd2c01924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.353398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8aac3f-fa7c-4d5a-8a0b-f66fd2c01924 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.353931 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8aac3f-fa7c-4d5a-8a0b-f66fd2c01924 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.354443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8aac3f-fa7c-4d5a-8a0b-f66fd2c01924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.406697 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c9179-5e74-428a-8af0-056de109c1db None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:44.432877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8aac3f-fa7c-4d5a-8a0b-f66fd2c01924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.434396 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8aac3f-fa7c-4d5a-8a0b-f66fd2c01924 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.442255 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.443067 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.443346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.443613 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.457281 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.458790 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:44.479129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:44.480593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:44.481614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:44.481614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:44.481614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:44.481614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:44.484941 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 278/529] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:44.485868 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.551886 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 252/530] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:44.552821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8aac3f-fa7c-4d5a-8a0b-f66fd2c01924 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.584846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525b1e-81c6-4c4c-a381-d2f37a529cd4 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:44.618457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.619385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.619664 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.619942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.620726 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:44.621031 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.622477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.640758 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 279/531] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:44.646343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.653663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8aac3f-fa7c-4d5a-8a0b-f66fd2c01924 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:44.688031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.688707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.688993 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.689301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.690043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:44.690355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.691734 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.704183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 253/532] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:44.705027 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.749984 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37780f-eb20-41b3-a0d5-c95a37b5df58 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:44.784388 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.785207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.785485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.785758 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.795828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.797303 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.806079 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c4a11-024f-401f-bc06-9a40f8b4e282 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:44.814737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:44.816871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:44.819270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:44.819270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:44.819270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:44.819270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:44.819270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 280/533] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:44.819270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.846280 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:44.846914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:44.847196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:44.847520 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:44.848369 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=14fc4922f2c844aabeba87c71be65b6f) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:44.848681 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:44.850177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:44.857441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 254/534] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/role_assignments?user.id=14fc4922f2c844aabeba87c71be65b6f&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:44.858334 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:44.975419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2470624-7a4b-452e-85fc-fa730891af61 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:45.012183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:45.012869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:45.013180 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:45.013454 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:45.014194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:45.014500 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:45.015900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:45.029628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 255/535] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:45.035227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:45.051275 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe44a2-16a9-47a6-9f63-125f90497368 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:45.140286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eb4a2-5803-4da0-b670-f65d823aee7d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:45.150555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:45.151366 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:45.151649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:45.151928 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:45.171217 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:45.172712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:45.195326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:45.197715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:45.200012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:45.200012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:45.200012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:45.200012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:45.200012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:45.200012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:45.200012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:45.200012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:45.205129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 281/536] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:45.209435 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:45.210238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:45.210514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:45.210780 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:45.212630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:45.227589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:45.229075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:45.265066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:45.267252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 256/537] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:45.269674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:45.348359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f6a8c-015e-448c-add4-d74931773fc1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:45.381160 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:45.381815 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:45.382094 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:45.382365 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:45.383702 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:45.384009 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:45.385443 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:45.394826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1756bfd-55cb-43ec-b557-c52ff6ea835f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:45.397611 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 282/538] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:45.403917 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:45.428787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:45.429481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:45.429763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:45.430083 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:45.430870 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:45.431179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:45.432663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:45.444974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 257/539] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:45.451780 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:45.521234 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf8a08-6e87-4593-8f9e-1a0b15bf269a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:45.561834 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21477bf-44ab-4315-8f33-e2e5e04e71cd None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:45.564741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf887cca-719d-4a70-bcdf-351c17440dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:45.565433 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf887cca-719d-4a70-bcdf-351c17440dfb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:45.565708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf887cca-719d-4a70-bcdf-351c17440dfb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:45.565976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf887cca-719d-4a70-bcdf-351c17440dfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:45.566809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf887cca-719d-4a70-bcdf-351c17440dfb None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=9006561a3f5340c9b397c4892f7d1f34) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:45.567115 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf887cca-719d-4a70-bcdf-351c17440dfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:45.569741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf887cca-719d-4a70-bcdf-351c17440dfb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:45.581901 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 283/540] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:45 2022] GET /identity/v3/role_assignments?user.id=9006561a3f5340c9b397c4892f7d1f34&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:45.641188 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c932b-01a5-4a26-877b-3ee7a1755652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:45.641924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c932b-01a5-4a26-877b-3ee7a1755652 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:45.642205 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c932b-01a5-4a26-877b-3ee7a1755652 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:45.642475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c932b-01a5-4a26-877b-3ee7a1755652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:45.643333 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c932b-01a5-4a26-877b-3ee7a1755652 None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=fb19da929a66435e86ed966bf4a3c9cb) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:45.643639 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c932b-01a5-4a26-877b-3ee7a1755652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:45.645208 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c932b-01a5-4a26-877b-3ee7a1755652 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:45.652459 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 258/541] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:45 2022] GET /identity/v3/role_assignments?user.id=fb19da929a66435e86ed966bf4a3c9cb&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:46.441089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53ae20-85a9-4f81-9530-83ffc52d0474 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:46.441089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53ae20-85a9-4f81-9530-83ffc52d0474 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:46.441089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53ae20-85a9-4f81-9530-83ffc52d0474 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:46.441089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 284/542] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:46.446880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ce9b3-6b2e-4c3d-9c19-dc886f02bd4c None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:46.447208 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ce9b3-6b2e-4c3d-9c19-dc886f02bd4c None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:46.447486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ce9b3-6b2e-4c3d-9c19-dc886f02bd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:46.474605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-614ce9b3-6b2e-4c3d-9c19-dc886f02bd4c None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:46.476727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-614ce9b3-6b2e-4c3d-9c19-dc886f02bd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:46.553611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 259/543] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:46.591826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53ae20-85a9-4f81-9530-83ffc52d0474 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:46.706742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53ae20-85a9-4f81-9530-83ffc52d0474 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:46.752078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d9f41-ba7b-43f9-a5a2-6780ac6705c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:46.752878 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d9f41-ba7b-43f9-a5a2-6780ac6705c2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:46.753180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d9f41-ba7b-43f9-a5a2-6780ac6705c2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:46.753452 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d9f41-ba7b-43f9-a5a2-6780ac6705c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:46.770016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d9f41-ba7b-43f9-a5a2-6780ac6705c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-441d9f41-ba7b-43f9-a5a2-6780ac6705c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:46.778319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:46.780595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:46.782714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 285/544] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:46.789079 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ce9b3-6b2e-4c3d-9c19-dc886f02bd4c None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:46.958149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ce9b3-6b2e-4c3d-9c19-dc886f02bd4c None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:46.993422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:46.994362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:46.994640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:46.994912 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:46.995657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] RBAC: Adding query filter params (name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:46.995962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:46.997342 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:46.998795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd793a0-15dd-478f-ab85-283b0a6c1eb0 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:46.999082 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd793a0-15dd-478f-ab85-283b0a6c1eb0 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:46.999362 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd793a0-15dd-478f-ab85-283b0a6c1eb0 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.001015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 286/545] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:47.006303 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 260/546] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:47.006545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2f469-5172-46a6-ab50-f1b1f2819355 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.006776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2f469-5172-46a6-ab50-f1b1f2819355 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.007123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2f469-5172-46a6-ab50-f1b1f2819355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.012652 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:47.038423 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-abd2f469-5172-46a6-ab50-f1b1f2819355 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:47.039675 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-abd2f469-5172-46a6-ab50-f1b1f2819355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:47.138448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 287/547] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:47.161469 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2f469-5172-46a6-ab50-f1b1f2819355 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:47.210202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9068f0a-0e6f-49d0-b1b7-394267fe9b5e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:47.280509 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:47.281371 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.281656 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.281930 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.294386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:47.295858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:47.314526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:47.315533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:47.316496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:47.316496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:47.316496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:47.316496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:47.316496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 261/548] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:47.323297 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:47.356775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2f469-5172-46a6-ab50-f1b1f2819355 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:47.418977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce46a5-1858-4c85-a029-0850831e2be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:47.419779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce46a5-1858-4c85-a029-0850831e2be6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.420055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce46a5-1858-4c85-a029-0850831e2be6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.420320 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce46a5-1858-4c85-a029-0850831e2be6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.437306 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce46a5-1858-4c85-a029-0850831e2be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-0fce46a5-1858-4c85-a029-0850831e2be6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:47.445086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:47.446645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:47.447670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:47.462180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 288/549] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:47.467456 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce46a5-1858-4c85-a029-0850831e2be6 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:47.510632 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077dd15-aac9-47a3-9910-cde504b89de5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:47.554595 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56948ee-9b4a-44e6-a72e-1885f02c921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:47.555571 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56948ee-9b4a-44e6-a72e-1885f02c921e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.555571 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56948ee-9b4a-44e6-a72e-1885f02c921e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.555926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56948ee-9b4a-44e6-a72e-1885f02c921e None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.556547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56948ee-9b4a-44e6-a72e-1885f02c921e None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:47.556849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56948ee-9b4a-44e6-a72e-1885f02c921e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:47.558238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56948ee-9b4a-44e6-a72e-1885f02c921e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:47.570970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 262/550] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:47.571944 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9490001-2e54-4f92-8d2d-ce4b070374f9 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.572227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9490001-2e54-4f92-8d2d-ce4b070374f9 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.572502 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9490001-2e54-4f92-8d2d-ce4b070374f9 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.578339 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 263/551] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:47.579072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9490001-2e54-4f92-8d2d-ce4b070374f9 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:47.607769 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce46a5-1858-4c85-a029-0850831e2be6 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:47.642947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0636630-9c57-4540-93cd-d8851b50dc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:47.643976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0636630-9c57-4540-93cd-d8851b50dc5f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.644524 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0636630-9c57-4540-93cd-d8851b50dc5f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.645019 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0636630-9c57-4540-93cd-d8851b50dc5f None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.646046 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0636630-9c57-4540-93cd-d8851b50dc5f None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:47.646562 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0636630-9c57-4540-93cd-d8851b50dc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:47.648217 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0636630-9c57-4540-93cd-d8851b50dc5f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:47.657659 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 289/552] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:47.659268 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2653d-c2a6-4f70-b28e-55d6b28d5f5e None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.659768 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2653d-c2a6-4f70-b28e-55d6b28d5f5e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.660266 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2653d-c2a6-4f70-b28e-55d6b28d5f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.677012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-10b2653d-c2a6-4f70-b28e-55d6b28d5f5e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:47.678608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-10b2653d-c2a6-4f70-b28e-55d6b28d5f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:47.679207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9490001-2e54-4f92-8d2d-ce4b070374f9 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:47.713925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:47.714635 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.714916 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.715187 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.722819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:47.724100 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:47.738763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:47.739843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:15:47.740881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:47.745153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 264/553] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:47.746000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:47.755024 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 290/554] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:47.756196 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2653d-c2a6-4f70-b28e-55d6b28d5f5e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:47.845477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f27c3-7be9-417d-938b-aee768c679ae None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:47.862859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2653d-c2a6-4f70-b28e-55d6b28d5f5e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:47.879546 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:47.880341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.880622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.880896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.891715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:47.893075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:47.901574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:47.902578 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:47.903105 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:47.903583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:47.904628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:47.905197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:47.906864 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:47.911913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:47.912997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:47.915651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:47.915651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:47.915651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:47.915651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:47.915651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 265/555] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:47.915651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:47.929792 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 291/556] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:47.933946 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.027355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819dc61-2338-4055-a5fb-bf82d014c294 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.036868 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bc2d0-ee4b-442a-a8a0-0a66a67bd724 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.061481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7306256-d03d-4d15-bfba-8618349deba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.062581 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306256-d03d-4d15-bfba-8618349deba8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.062864 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306256-d03d-4d15-bfba-8618349deba8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.063211 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7306256-d03d-4d15-bfba-8618349deba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.074170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7306256-d03d-4d15-bfba-8618349deba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:48.074486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.075388 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.075888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.076380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.077508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:48.078039 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.079611 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-a7306256-d03d-4d15-bfba-8618349deba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:48.081856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:48.082858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:48.083770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 266/557] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:48.084504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7306256-d03d-4d15-bfba-8618349deba8 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.095524 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 292/558] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:48.096729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.188114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7306256-d03d-4d15-bfba-8618349deba8 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.203581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2c019-fcdd-4f3f-9695-cdc2dfc8261f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.222169 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49c13f-e49b-4cb9-a63f-187da041d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.222828 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49c13f-e49b-4cb9-a63f-187da041d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.223112 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49c13f-e49b-4cb9-a63f-187da041d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.223389 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49c13f-e49b-4cb9-a63f-187da041d8b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.241935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.243165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.243445 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.243737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.245367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:48.245694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.247211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.258584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 293/559] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:48.259630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.302111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49c13f-e49b-4cb9-a63f-187da041d8b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.304004 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49c13f-e49b-4cb9-a63f-187da041d8b0 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.382053 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c5b3-4b4a-4076-bb98-fec0fb862616 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.411834 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 267/560] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:48.412960 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49c13f-e49b-4cb9-a63f-187da041d8b0 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.418850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66a4ba-13f3-44dd-b665-3a8dfbd299e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.419655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a4ba-13f3-44dd-b665-3a8dfbd299e6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.419938 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a4ba-13f3-44dd-b665-3a8dfbd299e6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.420214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66a4ba-13f3-44dd-b665-3a8dfbd299e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.427944 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66a4ba-13f3-44dd-b665-3a8dfbd299e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.429499 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66a4ba-13f3-44dd-b665-3a8dfbd299e6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-ab66a4ba-13f3-44dd-b665-3a8dfbd299e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:48.444007 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:48.445545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:15:48.446787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:48.449765 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 294/561] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:48.450998 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7d5d0-cda4-4879-9e10-67509dd96794 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.451288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7d5d0-cda4-4879-9e10-67509dd96794 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.451565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7d5d0-cda4-4879-9e10-67509dd96794 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.452979 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 295/562] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:48.456591 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7d5d0-cda4-4879-9e10-67509dd96794 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.512997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49c13f-e49b-4cb9-a63f-187da041d8b0 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.545151 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.545940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.546221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.546495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.551805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7d5d0-cda4-4879-9e10-67509dd96794 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.556916 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.558416 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:48.575436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:48.576560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:48.577769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:48.577769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:48.577769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:48.577769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:48.577769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 268/563] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:48.577769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.588071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6638-9c2a-43a6-8953-3685b43bfb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.588923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6638-9c2a-43a6-8953-3685b43bfb35 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.589231 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6638-9c2a-43a6-8953-3685b43bfb35 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.589499 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6638-9c2a-43a6-8953-3685b43bfb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.600519 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6638-9c2a-43a6-8953-3685b43bfb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.602540 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6638-9c2a-43a6-8953-3685b43bfb35 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-7e6b6638-9c2a-43a6-8953-3685b43bfb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:48.619800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:48.621309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:48.622460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:48.622460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:48.622460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:48.622460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:48.622460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 296/564] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:48.622460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff06b92-0314-4bce-9750-2b8611be0982 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.622460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff06b92-0314-4bce-9750-2b8611be0982 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.622919 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff06b92-0314-4bce-9750-2b8611be0982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.638335 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-cff06b92-0314-4bce-9750-2b8611be0982 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:48.639518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-cff06b92-0314-4bce-9750-2b8611be0982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:48.671762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4fb29-4f9d-45a5-a1e6-cf4c22fcd9a5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.702230 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 297/565] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:48.703446 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d6d10-188d-4166-ba09-59acd6a7d059 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.703788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d6d10-188d-4166-ba09-59acd6a7d059 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.704069 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d6d10-188d-4166-ba09-59acd6a7d059 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.704595 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.705500 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.705617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 298/566] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:48.705708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.706026 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.706135 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d6d10-188d-4166-ba09-59acd6a7d059 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.706970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:48.707335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.708819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.730955 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 269/567] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:48.730955 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.813288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d6d10-188d-4166-ba09-59acd6a7d059 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.819767 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11894f1d-498a-4b66-bdef-e4b82dca2a92 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.845341 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73241b-e088-42e0-8071-bfe8b6397622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.845996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73241b-e088-42e0-8071-bfe8b6397622 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.846281 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73241b-e088-42e0-8071-bfe8b6397622 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.846559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73241b-e088-42e0-8071-bfe8b6397622 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.847280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73241b-e088-42e0-8071-bfe8b6397622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:48.847589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73241b-e088-42e0-8071-bfe8b6397622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.848957 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73241b-e088-42e0-8071-bfe8b6397622 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.852695 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.853394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.853672 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.853942 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.854659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:48.854963 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:48.856302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:48.860204 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 299/568] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:48.861495 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629f89d-adc9-48ab-97ae-42d497ddbe2a None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.861791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629f89d-adc9-48ab-97ae-42d497ddbe2a None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.862071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629f89d-adc9-48ab-97ae-42d497ddbe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:48.867485 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 270/569] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:48.868272 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.881801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-1629f89d-adc9-48ab-97ae-42d497ddbe2a None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:48.883053 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-1629f89d-adc9-48ab-97ae-42d497ddbe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:48.946905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 300/570] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:48.947727 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629f89d-adc9-48ab-97ae-42d497ddbe2a None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:48.962140 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5522d-3d99-4c71-8a6c-465cfc0b8cd5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:48.993651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800bf9c-8fdf-408a-8680-a05c500f2574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:48.994789 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800bf9c-8fdf-408a-8680-a05c500f2574 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:48.995074 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800bf9c-8fdf-408a-8680-a05c500f2574 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:48.995346 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800bf9c-8fdf-408a-8680-a05c500f2574 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.005270 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800bf9c-8fdf-408a-8680-a05c500f2574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-1800bf9c-8fdf-408a-8680-a05c500f2574 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:49.013498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:49.014684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:49.015591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 271/571] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:49.016338 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800bf9c-8fdf-408a-8680-a05c500f2574 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.048468 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629f89d-adc9-48ab-97ae-42d497ddbe2a None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.084624 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.085383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.085665 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.085933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.095676 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.097194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.114525 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800bf9c-8fdf-408a-8680-a05c500f2574 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:49.115816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:49.116937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:49.119100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:49.119100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:49.119100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:49.119100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:49.119100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 301/572] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:49.119100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.152182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.152972 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.153283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.153560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] PATH_INFO: `/v3/users/glance` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.160850 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.162314 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:49.176832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:49.178029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 272/573] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:49.184514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.212274 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869eede-b6ef-4083-9150-f16905fd0733 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.244820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.245626 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.245907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.246180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.255133 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.256818 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:49.270503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:49.271610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 302/574] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:49.272581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.276476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42e060-f5d0-49c2-b8e3-ad182ccafecc None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.309190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e814f1-599c-42c9-8255-332c3da15ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.309972 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e814f1-599c-42c9-8255-332c3da15ddf None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.310254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e814f1-599c-42c9-8255-332c3da15ddf None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.310548 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e814f1-599c-42c9-8255-332c3da15ddf None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.320265 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e814f1-599c-42c9-8255-332c3da15ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f9e814f1-599c-42c9-8255-332c3da15ddf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:49.328033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:49.329328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 273/575] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:49.330426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e814f1-599c-42c9-8255-332c3da15ddf None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.369731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c32262-0843-4f84-8aed-e3ca353bd128 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.400439 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.401132 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.401418 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.401694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.402444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:49.402753 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.404113 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.411795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 303/576] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:49.412492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.431950 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e814f1-599c-42c9-8255-332c3da15ddf None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.463561 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.464215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.464497 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.464769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.465526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:49.465832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.467186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.479024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 274/577] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:49.479634 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.506805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b8f1a-6b0f-4652-bf91-ae1bf5dbf10d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.538330 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.538994 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.539903 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.540184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.541013 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:49.541354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.542773 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.559028 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 304/578] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:49.559739 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.571611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b88d4-17cf-47d6-89ac-366d0b9435e1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.614962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.615651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.615935 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.616215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.616990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:49.617333 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.618748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.629947 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 275/579] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:49.630661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.660319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58b4b3-0b81-4952-aa02-26cb0e323aa6 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.691945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.692599 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.692880 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.693186 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.693926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:49.694228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.695573 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.703338 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 305/580] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:49.703872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.723982 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fe12f-7b75-41e7-81af-5178bff1dbcc None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.755877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.756657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.756937 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.757242 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.767170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.768608 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:49.787203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:49.788379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:49.790393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:49.790393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:49.790393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:49.790393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:49.790393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 276/581] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:49.790393 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.801243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c638cc-cd07-4eaf-9792-76688de2d295 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.832662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.833484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.833764 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.834032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.840931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.842368 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:49.856078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:49.857202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 306/582] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:49.858395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.887351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5952f1-0936-4f8e-9fc9-66b604268cd5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.919403 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.920180 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.920458 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.920726 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.930930 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.932364 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:49.952981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:49.954223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:49.955341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:49.955341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:49.955341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:49.955341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:49.955582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2f0af-bb54-4b0d-b88f-599143f38c66 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:49.955661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 277/583] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:49.955661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:49.987319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:49.987965 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:49.988241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:49.988510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:49.989361 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=e81aea4ac1ae45e2997c6c70e8c0028a) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:49.989669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:49.991326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:49.997019 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 307/584] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/role_assignments?user.id=e81aea4ac1ae45e2997c6c70e8c0028a&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:49.997881 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.066928 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3383f-8840-45df-adb4-00cb3c8c219a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.103973 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.104773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.105072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.105350 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.108302 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceefa42-3c96-4ad8-b758-d490d2724dcc None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.117862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.119622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:50.137775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:50.138813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:50.139783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:50.139783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:50.139783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:50.139783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:50.143604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 278/585] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:50.144398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.146602 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.147286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.147567 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.147843 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.149264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:50.149617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.151037 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.168334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 308/586] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:50.168334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.259691 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278fbe5-5205-45db-9319-71796725bd02 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.266525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747686f1-fb2b-4cdf-9f10-78ba5e03ab8a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.295735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.296399 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.296683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.296962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.297772 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:50.298199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.299639 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.301861 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.302521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.302802 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.303077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.303799 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:50.304105 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.305490 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.312230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 279/587] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:50.313072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.320607 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 309/588] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:50.322130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.410341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf936fa-c568-4e4c-a158-43970c492d57 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.424337 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df392437-94c9-4f5f-b02e-d487c9d3d82f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.444238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.444908 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.445240 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.445551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.446282 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:50.446618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.447953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.456224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.457016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.457318 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.457584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.459525 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 280/589] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:50.460454 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.465747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.467191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:50.481935 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:50.482980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:15:50.483999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:50.487796 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 310/590] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:50.489080 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.561446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a5b4f-46a4-4669-8a53-a5c2a3d2e17b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.583090 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555b2da-e744-49d7-b8d9-843e8c97f14e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.594884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b25fa-064b-4c67-a1bf-eb42f5677d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.595777 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b25fa-064b-4c67-a1bf-eb42f5677d7e None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.596061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b25fa-064b-4c67-a1bf-eb42f5677d7e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.596340 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b25fa-064b-4c67-a1bf-eb42f5677d7e None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/f3a73bc43c23400d8ee7d4e880aa7242/roles/05908f87badf453aaf2ee23a431445f0` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.611009 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b25fa-064b-4c67-a1bf-eb42f5677d7e None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=f3a73bc43c23400d8ee7d4e880aa7242, role_id=05908f87badf453aaf2ee23a431445f0)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.612446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b25fa-064b-4c67-a1bf-eb42f5677d7e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.617053 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.617975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.618253 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.618676 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.625243 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 281/591] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:50 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/f3a73bc43c23400d8ee7d4e880aa7242/roles/05908f87badf453aaf2ee23a431445f0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:50.626593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.626867 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b25fa-064b-4c67-a1bf-eb42f5677d7e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.628147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:50.643820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:50.644859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:50.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:50.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:50.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:50.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:50.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:50.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:50.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:50.645863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:50.650884 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 311/592] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:50.651885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b25fa-064b-4c67-a1bf-eb42f5677d7e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.762044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202954f-6db1-4faa-b34d-ede5bc632814 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.800124 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.800393 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.800811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.801117 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.801390 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.802228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.802344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:50.802533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.803017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.803498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.803995 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.812120 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.813960 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.824507 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 312/593] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:50.825536 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:50.830452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:50.831596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:15:50.832641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:50.836884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 282/594] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:50.837781 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.924860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358f6da-43e3-4cd6-af68-e86724ed0068 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.938970 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8cec6-dd1a-4a55-be4f-dbfb214947f3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:50.959328 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.959986 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.960262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.960535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.961383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:50.961689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.963106 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:50.973693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:50.974550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:50.974758 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 313/595] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:50.974932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:50.975138 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:50.980791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:50.985781 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:50.987247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:51.005568 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:51.007709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:51.010555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:51.010555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:51.010555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:51.010555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:51.010555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 283/596] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:51.010555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.081427 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb35c8-0f9e-41d5-af5c-d98a25ca1291 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.108593 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a188252-6905-46dd-871e-5baf06881c74 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.115958 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.116631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.116911 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.119072 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.119948 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:51.120261 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:51.121752 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.139925 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 314/597] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:15:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:51.140770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.146831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4e791-f740-41ac-8fc5-3da232d2820c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.147730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4e791-f740-41ac-8fc5-3da232d2820c None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.148010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4e791-f740-41ac-8fc5-3da232d2820c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.148285 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4e791-f740-41ac-8fc5-3da232d2820c None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/14fc4922f2c844aabeba87c71be65b6f/roles/05908f87badf453aaf2ee23a431445f0` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.163425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4e791-f740-41ac-8fc5-3da232d2820c None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=14fc4922f2c844aabeba87c71be65b6f, role_id=05908f87badf453aaf2ee23a431445f0)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:51.164897 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4e791-f740-41ac-8fc5-3da232d2820c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.182525 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 284/598] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:51 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/14fc4922f2c844aabeba87c71be65b6f/roles/05908f87badf453aaf2ee23a431445f0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:51.183871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4e791-f740-41ac-8fc5-3da232d2820c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.329078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4e791-f740-41ac-8fc5-3da232d2820c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.347448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a332ac-7f9c-4500-96c3-90a150a63b7d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.370034 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.370849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.371132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.371791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.384167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:51.385775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:51.407045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:51.408252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:51.409463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:51.409463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:51.409463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:51.409463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:51.409463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 285/599] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:51.415248 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.424835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.425820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.426253 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.426683 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.427621 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:51.428084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:51.429690 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.444487 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 315/600] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:51.449979 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.524250 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6b8e-9da4-4bdc-b776-89efd5a00260 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.551317 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6974c-fbdc-4a8a-b3d7-9a543a63e7da None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.558649 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.559284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.559669 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.560049 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.560905 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:51.561420 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:51.562949 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.575248 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 286/601] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:51.585181 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.587773 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.588846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.589329 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.589791 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.597179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:51.598998 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:51.614177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:51.615147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:51.616179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:51.616179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:51.616179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:51.616179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:51.616179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:51.616179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:51.616179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:51.616179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:51.618993 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 316/602] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:51.630005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.688865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148af22-03ac-4c7e-ab0e-9438b473b961 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.731165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db140742-fb83-438f-9230-c582b158a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.731968 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db140742-fb83-438f-9230-c582b158a010 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.732253 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db140742-fb83-438f-9230-c582b158a010 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.732529 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db140742-fb83-438f-9230-c582b158a010 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.749341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db140742-fb83-438f-9230-c582b158a010 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:51.750786 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db140742-fb83-438f-9230-c582b158a010 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-db140742-fb83-438f-9230-c582b158a010 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:51.781351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:51.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:51.783763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:51.783763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:51.783763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:51.783763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:51.783763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:51.783763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:51.783763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:51.783763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:51.789226 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 287/603] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:51.795025 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db140742-fb83-438f-9230-c582b158a010 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.796356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdead26-1757-4797-ad9b-0af519692480 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.882366 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.883835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.884295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.884726 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.885700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:51.886152 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:51.887747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:51.908780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 317/604] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:51.914992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:51.940999 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db140742-fb83-438f-9230-c582b158a010 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:51.996824 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:51.997557 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:51.997836 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:51.998108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:51.998902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:51.999209 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:52.000690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:52.013673 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 288/605] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:52.019088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:52.048692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb07f0f-9665-4888-9b6a-7a254ac8664b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:52.128529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f88cb-dba1-4e0c-941e-dcec8cf77d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:52.129540 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f88cb-dba1-4e0c-941e-dcec8cf77d04 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:52.129819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f88cb-dba1-4e0c-941e-dcec8cf77d04 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:52.130094 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f88cb-dba1-4e0c-941e-dcec8cf77d04 None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/9006561a3f5340c9b397c4892f7d1f34/roles/05908f87badf453aaf2ee23a431445f0` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:52.138703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ee866-6c80-4bef-b57f-c5a4c158382c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:52.163237 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f88cb-dba1-4e0c-941e-dcec8cf77d04 None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=9006561a3f5340c9b397c4892f7d1f34, role_id=05908f87badf453aaf2ee23a431445f0)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:52.165273 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f88cb-dba1-4e0c-941e-dcec8cf77d04 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:52.176095 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71eebfe-56db-41e8-a314-2e02e71a004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:52.177339 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eebfe-56db-41e8-a314-2e02e71a004c None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:52.177525 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eebfe-56db-41e8-a314-2e02e71a004c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:52.177769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71eebfe-56db-41e8-a314-2e02e71a004c None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/fb19da929a66435e86ed966bf4a3c9cb/roles/05908f87badf453aaf2ee23a431445f0` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:52.194983 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 318/606] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:51 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/9006561a3f5340c9b397c4892f7d1f34/roles/05908f87badf453aaf2ee23a431445f0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:15:52.196905 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71eebfe-56db-41e8-a314-2e02e71a004c None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=fb19da929a66435e86ed966bf4a3c9cb, role_id=05908f87badf453aaf2ee23a431445f0)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:52.198451 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71eebfe-56db-41e8-a314-2e02e71a004c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:52.227135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 289/607] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:52 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/fb19da929a66435e86ed966bf4a3c9cb/roles/05908f87badf453aaf2ee23a431445f0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:15:52.628940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bf46f-eb6e-4d11-b07e-52c2a2a563f5 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:52.629394 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bf46f-eb6e-4d11-b07e-52c2a2a563f5 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:52.629517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bf46f-eb6e-4d11-b07e-52c2a2a563f5 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:52.631171 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 319/608] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:52.636511 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87385c-da34-4e49-b6bb-328c8d99de0b None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:52.636799 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87385c-da34-4e49-b6bb-328c8d99de0b None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:52.637112 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87385c-da34-4e49-b6bb-328c8d99de0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:52.654302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-6e87385c-da34-4e49-b6bb-328c8d99de0b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:52.655489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-6e87385c-da34-4e49-b6bb-328c8d99de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:52.723454 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 290/609] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:52.746292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bf46f-eb6e-4d11-b07e-52c2a2a563f5 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:52.847965 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bf46f-eb6e-4d11-b07e-52c2a2a563f5 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:52.881466 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a998cb3-d74a-4a04-bb18-377673186f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:52.882180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998cb3-d74a-4a04-bb18-377673186f9d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:52.882458 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998cb3-d74a-4a04-bb18-377673186f9d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:52.882730 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998cb3-d74a-4a04-bb18-377673186f9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:52.893404 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a998cb3-d74a-4a04-bb18-377673186f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-2a998cb3-d74a-4a04-bb18-377673186f9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:52.901587 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:52.902687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:52.903684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 320/610] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:52.908803 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87385c-da34-4e49-b6bb-328c8d99de0b None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:53.006584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87385c-da34-4e49-b6bb-328c8d99de0b None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:53.054627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:53.055332 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:53.055612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:53.055884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:53.056640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] RBAC: Adding query filter params (name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:53.056940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:53.058322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:53.072654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 291/611] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:53.078184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a998cb3-d74a-4a04-bb18-377673186f9d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:53.189448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a998cb3-d74a-4a04-bb18-377673186f9d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:53.223423 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc37b9e-fed1-41c9-a3de-dfd1d1335b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:53.224221 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc37b9e-fed1-41c9-a3de-dfd1d1335b1c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:53.224501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc37b9e-fed1-41c9-a3de-dfd1d1335b1c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:53.224771 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc37b9e-fed1-41c9-a3de-dfd1d1335b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:53.237645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc37b9e-fed1-41c9-a3de-dfd1d1335b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:53.239114 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc37b9e-fed1-41c9-a3de-dfd1d1335b1c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-0bc37b9e-fed1-41c9-a3de-dfd1d1335b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:53.262766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:53.267276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:53.274185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:53.274185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:53.274185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:53.274185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:53.274185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 321/612] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:53.274905 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:53.401345 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bd06-12b6-4f7e-a60d-c21231d06dde None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:53.452723 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:53.453804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:53.454327 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:53.454837 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:53.455786 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:53.456313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:53.457945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:53.459265 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4acf4f-e979-401c-9fb4-26400e8e8c4d None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:53.459556 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4acf4f-e979-401c-9fb4-26400e8e8c4d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:53.459834 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4acf4f-e979-401c-9fb4-26400e8e8c4d None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:53.465060 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 322/613] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:53.466962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719677-3264-4e25-924a-2eeee603c39d None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:53.467247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719677-3264-4e25-924a-2eeee603c39d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:53.467524 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719677-3264-4e25-924a-2eeee603c39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:53.477873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 292/614] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:53.484791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:53.489723 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-81719677-3264-4e25-924a-2eeee603c39d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:53.490947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-81719677-3264-4e25-924a-2eeee603c39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:53.591625 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 323/615] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:15:53.627387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb322a05-ae72-42ef-8655-5ce3d73fb422 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:53.639936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81719677-3264-4e25-924a-2eeee603c39d None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:53.709893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:53.710645 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:53.711003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:53.711784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] PATH_INFO: `/v3/users/swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:53.730415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:53.731991 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:53.742805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81719677-3264-4e25-924a-2eeee603c39d None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:53.770163 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:53.771273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:53.772387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 293/616] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:53.779496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:53.783453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ac7d-57f0-4b59-8075-05ed4209728f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:53.784356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ac7d-57f0-4b59-8075-05ed4209728f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:53.784683 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ac7d-57f0-4b59-8075-05ed4209728f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:53.784963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ac7d-57f0-4b59-8075-05ed4209728f None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:53.802519 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80ac7d-57f0-4b59-8075-05ed4209728f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-9a80ac7d-57f0-4b59-8075-05ed4209728f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:53.810560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:53.812865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:53.814970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 324/617] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:15:53.822108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ac7d-57f0-4b59-8075-05ed4209728f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:53.959074 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9cea8-fef9-47c4-aac2-5a73ba908789 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:53.983262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ac7d-57f0-4b59-8075-05ed4209728f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:53.997455 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd39df-628c-482c-874f-7fecbedaf6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:53.998126 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd39df-628c-482c-874f-7fecbedaf6fd None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:53.998404 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd39df-628c-482c-874f-7fecbedaf6fd None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:53.998677 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd39df-628c-482c-874f-7fecbedaf6fd None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:53.999514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd39df-628c-482c-874f-7fecbedaf6fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:53.999819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd39df-628c-482c-874f-7fecbedaf6fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.001264 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd39df-628c-482c-874f-7fecbedaf6fd None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.019201 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 294/618] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:15:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:54.020588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723e048-b47e-4dd7-b062-edfcb234a1c3 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.020876 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723e048-b47e-4dd7-b062-edfcb234a1c3 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.021175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723e048-b47e-4dd7-b062-edfcb234a1c3 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.022538 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 295/619] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:54.027402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723e048-b47e-4dd7-b062-edfcb234a1c3 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.028000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24d90-f407-4cdb-89b3-3776171797bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.028680 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24d90-f407-4cdb-89b3-3776171797bb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.028963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24d90-f407-4cdb-89b3-3776171797bb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.029274 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24d90-f407-4cdb-89b3-3776171797bb None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.030250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24d90-f407-4cdb-89b3-3776171797bb None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:54.030562 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24d90-f407-4cdb-89b3-3776171797bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.032838 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24d90-f407-4cdb-89b3-3776171797bb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.043605 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 325/620] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:15:54.045310 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a01977-49f6-45e7-93d8-8e076369123c None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.045602 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a01977-49f6-45e7-93d8-8e076369123c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.046054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a01977-49f6-45e7-93d8-8e076369123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.066929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-b7a01977-49f6-45e7-93d8-8e076369123c None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:54.068126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-b7a01977-49f6-45e7-93d8-8e076369123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:54.134156 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723e048-b47e-4dd7-b062-edfcb234a1c3 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.153300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 326/621] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:54.154192 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a01977-49f6-45e7-93d8-8e076369123c None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.171651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.172750 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.173061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.173621 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.185733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.187182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:54.210119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:54.212017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:54.213276 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:54.213276 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:54.213276 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:54.213276 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:54.216282 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 296/622] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:54.217237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.277221 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a01977-49f6-45e7-93d8-8e076369123c None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.310910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.312116 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.312611 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.313130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.325703 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.327344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.341990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becae4-cf29-44c5-ab6e-c17f8e1ab00a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:54.349111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:54.350057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:54.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:54.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:54.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:54.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:54.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 327/623] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:54.352551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.382223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4937f-8f19-4854-a886-8a667ec623af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.383476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4937f-8f19-4854-a886-8a667ec623af None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.384016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4937f-8f19-4854-a886-8a667ec623af None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.384532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4937f-8f19-4854-a886-8a667ec623af None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.395586 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4937f-8f19-4854-a886-8a667ec623af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-30e4937f-8f19-4854-a886-8a667ec623af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:54.404957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:54.406121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:54.407023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:54.416133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 297/624] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:54.417380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4937f-8f19-4854-a886-8a667ec623af None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.472074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6beff-b491-44e2-a6ab-830e605f672d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.519106 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.519699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.519983 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.520261 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.521488 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:54.521936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.523320 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.524167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4937f-8f19-4854-a886-8a667ec623af None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.541125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 328/625] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:54.542326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.563764 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.564808 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.565360 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.565860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.566869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:54.567397 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.568995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.581398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 298/626] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:54.582573 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.673435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0a3a0-7ea6-4381-9550-7ec3c683777e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.684209 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cbc07-2716-42c9-aabb-dbe5b491563b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.707838 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.708927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.709502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.710011 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.711535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:54.712058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.713676 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.726209 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.726209 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.726209 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.727311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.727529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 329/627] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:54.728566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.735375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.736824 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:54.755043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:54.756039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:54.756995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:54.756995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:54.756995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:54.756995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:54.756995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:54.756995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:54.756995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:54.756995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:54.761312 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 299/628] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:54.762504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.842835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121536f-74c8-42c9-93b3-4092b44b9341 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.870440 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20278798-f89e-4d01-9e28-85058460cb70 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:54.877693 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.878778 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.879290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.879767 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.887813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.889665 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:54.904463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:54.905535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:15:54.906558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:54.907146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:54.907146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:54.907146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:54.907146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:54.910779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 330/629] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:54.911950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:54.918597 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:54.920060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:54.937408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:54.938437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:54.939386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:54.939386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:54.939386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:54.939386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:54.943198 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 300/630] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:54.944052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.015026 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433b18a-03e5-4bbb-9789-2904f8239db0 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.040991 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86250e16-6f8d-4f32-852f-4b4ed2042f38 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.047606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.048344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.048629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.048902 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.049729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:55.050037 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:55.051461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:55.062905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 331/631] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:55.063695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.077609 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db7a9c-0594-4939-9469-fad9286b05c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.078273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db7a9c-0594-4939-9469-fad9286b05c7 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.078551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db7a9c-0594-4939-9469-fad9286b05c7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.078830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db7a9c-0594-4939-9469-fad9286b05c7 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.079654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db7a9c-0594-4939-9469-fad9286b05c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:55.079959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db7a9c-0594-4939-9469-fad9286b05c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:55.081401 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db7a9c-0594-4939-9469-fad9286b05c7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:55.098644 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 301/632] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:15:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:55.099785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1b8b-94f9-4461-a8b1-a627dad2a44d None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.100064 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1b8b-94f9-4461-a8b1-a627dad2a44d None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.100395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1b8b-94f9-4461-a8b1-a627dad2a44d None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.105902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 302/633] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:55.105902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89318326-fdd0-4257-8783-9d81a1764808 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.105902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89318326-fdd0-4257-8783-9d81a1764808 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.105902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89318326-fdd0-4257-8783-9d81a1764808 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.105902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 303/634] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:55.105902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89318326-fdd0-4257-8783-9d81a1764808 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.161760 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca3f62-98e9-47ff-90ab-80a74db6449e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.194094 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.194747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.195036 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.195334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.196052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:55.196356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:55.197742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:55.201825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89318326-fdd0-4257-8783-9d81a1764808 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.208267 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 332/635] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:55.209067 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.238299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862f997-3adc-4bc7-8739-f36b2de293d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.239173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f997-3adc-4bc7-8739-f36b2de293d0 None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.239450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f997-3adc-4bc7-8739-f36b2de293d0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.239721 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862f997-3adc-4bc7-8739-f36b2de293d0 None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/e81aea4ac1ae45e2997c6c70e8c0028a/roles/05908f87badf453aaf2ee23a431445f0` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.257067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862f997-3adc-4bc7-8739-f36b2de293d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=e81aea4ac1ae45e2997c6c70e8c0028a, role_id=05908f87badf453aaf2ee23a431445f0)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:55.258551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862f997-3adc-4bc7-8739-f36b2de293d0 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:55.270522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 304/636] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:15:55 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/e81aea4ac1ae45e2997c6c70e8c0028a/roles/05908f87badf453aaf2ee23a431445f0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:15:55.271985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6caac9-4c9b-4393-8457-b97b210cb77a None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.272269 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6caac9-4c9b-4393-8457-b97b210cb77a None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.272542 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6caac9-4c9b-4393-8457-b97b210cb77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.297956 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-7b6caac9-4c9b-4393-8457-b97b210cb77a None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:55.299880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-7b6caac9-4c9b-4393-8457-b97b210cb77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:55.308539 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea933baa-1341-4ca7-847a-d67b09d3c602 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.344286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd20021-dc4b-491f-9689-5573ec5f62e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.345417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd20021-dc4b-491f-9689-5573ec5f62e1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.345907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd20021-dc4b-491f-9689-5573ec5f62e1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.346369 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd20021-dc4b-491f-9689-5573ec5f62e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.381149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 305/637] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:55.381149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00055dc-239b-4295-8825-72fb924fcd49 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.381149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00055dc-239b-4295-8825-72fb924fcd49 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.381149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00055dc-239b-4295-8825-72fb924fcd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.398269 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-d00055dc-239b-4295-8825-72fb924fcd49 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:55.399499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-d00055dc-239b-4295-8825-72fb924fcd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:55.439765 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd20021-dc4b-491f-9689-5573ec5f62e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:55.441292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd20021-dc4b-491f-9689-5573ec5f62e1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:55.475966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 306/638] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:55.477468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00055dc-239b-4295-8825-72fb924fcd49 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.547543 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 333/639] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:55.548458 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd20021-dc4b-491f-9689-5573ec5f62e1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.592484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00055dc-239b-4295-8825-72fb924fcd49 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.625790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.626589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.626868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.627182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] PATH_INFO: `/v3/users/glance` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.634860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:55.636248 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:55.644436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd20021-dc4b-491f-9689-5573ec5f62e1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:55.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:55.651484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:15:55.652518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:55.658810 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 307/640] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:55.658810 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.678480 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1a721-5d24-4878-b601-b01eb9a65783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.679319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1a721-5d24-4878-b601-b01eb9a65783 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.679599 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1a721-5d24-4878-b601-b01eb9a65783 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.680675 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1a721-5d24-4878-b601-b01eb9a65783 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.690212 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1a721-5d24-4878-b601-b01eb9a65783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a9d1a721-5d24-4878-b601-b01eb9a65783 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:55.698511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:55.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:55.700448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:55.704414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 334/641] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:55.705258 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1a721-5d24-4878-b601-b01eb9a65783 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.762804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82dd6-9956-485f-8e34-6c03a57233db None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.797222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c33d8-f535-4d39-9e01-dd01d0a8f303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.798022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c33d8-f535-4d39-9e01-dd01d0a8f303 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.798308 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c33d8-f535-4d39-9e01-dd01d0a8f303 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.798586 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c33d8-f535-4d39-9e01-dd01d0a8f303 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.802666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1a721-5d24-4878-b601-b01eb9a65783 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.809142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c33d8-f535-4d39-9e01-dd01d0a8f303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f33c33d8-f535-4d39-9e01-dd01d0a8f303 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:55.816926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:55.818085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 308/642] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:55.824560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c33d8-f535-4d39-9e01-dd01d0a8f303 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.841570 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.842367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.842645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.842914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.853913 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:55.855385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:55.873001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:55.874173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:55.875461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:55.875461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:55.875461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:55.875461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:55.878852 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 335/643] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:55.879693 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:55.926476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c33d8-f535-4d39-9e01-dd01d0a8f303 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.967896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:55.968583 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:55.968873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:55.969182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:55.970032 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:55.970340 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:55.971786 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:55.981016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242b37d-1639-478b-9419-9d5d31a76872 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:55.988880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 309/644] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:55.989955 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.018875 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.019546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.019817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.020087 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.020846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:56.021177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.022522 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.036202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 336/645] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:56.036202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.094850 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a34047-e13b-4f4d-92c7-9971f30f239d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.129938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.130608 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.130885 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.131203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.132041 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:56.132223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.134107 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.137840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a47f38-5e09-46bb-a625-4d18a88571f2 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.143398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 310/646] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:56.144875 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.171969 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.172968 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.173445 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.173737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.174519 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:56.174759 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.176208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.193191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 337/647] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:56.193733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.251497 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12506314-f155-4024-95db-83bdf9ff2fc7 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.286710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b339f-85c2-44fa-83c8-218c246055d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.287365 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b339f-85c2-44fa-83c8-218c246055d5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.287652 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b339f-85c2-44fa-83c8-218c246055d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.287937 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b339f-85c2-44fa-83c8-218c246055d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.296134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a881a5-582b-4411-8dd7-c32b2c189b56 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.334510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.335295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.335568 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.335882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.345561 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.347005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.354475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b339f-85c2-44fa-83c8-218c246055d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.355906 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b339f-85c2-44fa-83c8-218c246055d5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:56.364148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:56.370630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:56.371885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:56.371885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:56.371885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:56.371885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:56.371885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 338/648] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:56.371885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.461730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 311/649] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:56.462634 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b339f-85c2-44fa-83c8-218c246055d5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.472305 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f92d7d-f189-449a-8ea4-e7c4029b1678 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.510885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.512066 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.512689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.512976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.525999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.527945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:56.545722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:56.546933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:56.547908 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:56.547908 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:56.547908 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:56.547908 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:56.547908 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 339/650] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:56.548231 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.569105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b339f-85c2-44fa-83c8-218c246055d5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.608190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.609321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.609612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.609891 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.618426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.620545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:56.635035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:56.636111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 312/651] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:56.637228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.646379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb15bec-8a43-4580-bb48-0755c90eef32 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.679440 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.680099 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.680381 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.680660 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.681413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:56.681721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.683076 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.698548 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 340/652] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:56.698548 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.741143 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7806f-000f-4bfb-b044-126d6c26c5f0 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.774077 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.774854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.775251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.775527 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.785596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.787022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.790960 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bad8f4-ea82-4dd3-a183-02bc1cceb1a8 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:56.804330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:56.805388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:56.807019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:56.807019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:56.807019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:56.807019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:56.807019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 313/653] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:56.807019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.825780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.826447 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.826725 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.827001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.827733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:56.828029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.829436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.846004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 341/654] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:56.846004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.916394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ae003-fd6c-4aa0-af4d-2e6510462b18 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.941679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863b64e-6772-4af9-94f6-312dfb1e9b38 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:56.950647 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.951804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.952264 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.952545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.953374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:56.953684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.955623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:56.975604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 314/655] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:56.975604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:56.983389 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:56.984276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:56.984565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:56.984836 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:56.992572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:56.994242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:57.009169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:57.010229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 342/656] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:57.011269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.084710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bf36c-d4a6-4ef8-b793-6eceb112df63 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.111794 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4dd55-c072-4dc0-a9a5-7d72064bd0a9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.117732 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.118399 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.118931 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.119208 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.119938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:57.120246 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.121636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.138152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 315/657] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:57.138152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.151814 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.152611 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.152888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.153191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.160286 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.162515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:57.177472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:57.178545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:15:57.180077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:57.183645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 343/658] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:57.184544 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.235195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5ca0c-c548-4963-8b6b-526a644e18f5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.269128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.269784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.270072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.270343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.271169 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=f3a73bc43c23400d8ee7d4e880aa7242) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:57.271483 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.272900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.285103 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 316/659] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/role_assignments?user.id=f3a73bc43c23400d8ee7d4e880aa7242&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:57.285103 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.299837 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517d13d-c63c-4132-977b-8d17c1615f45 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.350870 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.352338 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.352809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.353546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.354774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:57.355154 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.361317 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.390725 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 344/660] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:57.391787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.401268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48deb8-6cbc-45ff-9183-2d9efc979a1c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.438065 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.438953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.439237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.439512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.447934 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.449577 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:57.468376 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:57.469591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:57.470582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 317/661] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:57.471111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.519706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e68d-d295-4a5d-8269-2c4b042f3f6a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.559358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.560029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.560314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.560586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.561469 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:57.561779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.563211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.580952 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79f78e-85d0-4f14-abd7-f232d60b3809 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.583848 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 345/662] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:57.583993 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.616821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbb1d1-1c99-4fb3-96f1-9adab6ffc3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.618114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbb1d1-1c99-4fb3-96f1-9adab6ffc3e7 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.618400 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbb1d1-1c99-4fb3-96f1-9adab6ffc3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbb1d1-1c99-4fb3-96f1-9adab6ffc3e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.683454 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbb1d1-1c99-4fb3-96f1-9adab6ffc3e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.684904 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbb1d1-1c99-4fb3-96f1-9adab6ffc3e7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.685383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb557-e18d-4c25-a0b0-38a745fe22e1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.719531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfaac9-89c4-4ed2-868e-08b63d9e54bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.720202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfaac9-89c4-4ed2-868e-08b63d9e54bc None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.720484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfaac9-89c4-4ed2-868e-08b63d9e54bc None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.720756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfaac9-89c4-4ed2-868e-08b63d9e54bc None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.721614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfaac9-89c4-4ed2-868e-08b63d9e54bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:57.721894 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfaac9-89c4-4ed2-868e-08b63d9e54bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.723343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfaac9-89c4-4ed2-868e-08b63d9e54bc None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.740242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 346/663] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:57.740242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81205-893e-4ffb-b777-997695024e63 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.740242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81205-893e-4ffb-b777-997695024e63 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.740242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d81205-893e-4ffb-b777-997695024e63 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.740242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 347/664] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:57.740242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d81205-893e-4ffb-b777-997695024e63 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.805661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 318/665] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:57.805661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbb1d1-1c99-4fb3-96f1-9adab6ffc3e7 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:57.848255 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d81205-893e-4ffb-b777-997695024e63 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.881300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895020b-9ebe-469e-a135-2550acade95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.881952 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895020b-9ebe-469e-a135-2550acade95b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.882233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895020b-9ebe-469e-a135-2550acade95b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.882510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895020b-9ebe-469e-a135-2550acade95b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.902462 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbb1d1-1c99-4fb3-96f1-9adab6ffc3e7 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:57.937195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954034e3-9471-4fc6-96d6-11474df5020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:57.937861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954034e3-9471-4fc6-96d6-11474df5020b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.938148 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954034e3-9471-4fc6-96d6-11474df5020b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.938428 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954034e3-9471-4fc6-96d6-11474df5020b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.939280 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954034e3-9471-4fc6-96d6-11474df5020b None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=14fc4922f2c844aabeba87c71be65b6f) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:57.939565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954034e3-9471-4fc6-96d6-11474df5020b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.940988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954034e3-9471-4fc6-96d6-11474df5020b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.953247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 319/666] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/role_assignments?user.id=14fc4922f2c844aabeba87c71be65b6f&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:57.953247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0245a-3398-4c35-b96d-2fae09b1fb3e None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:57.953247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0245a-3398-4c35-b96d-2fae09b1fb3e None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:57.953247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0245a-3398-4c35-b96d-2fae09b1fb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:57.953679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895020b-9ebe-469e-a135-2550acade95b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:57.953679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895020b-9ebe-469e-a135-2550acade95b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:57.978188 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-70b0245a-3398-4c35-b96d-2fae09b1fb3e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:57.979439 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-70b0245a-3398-4c35-b96d-2fae09b1fb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:58.054120 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 320/667] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:15:58.055064 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0245a-3398-4c35-b96d-2fae09b1fb3e None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.067191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 348/668] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:15:58.076246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895020b-9ebe-469e-a135-2550acade95b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.165088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0245a-3398-4c35-b96d-2fae09b1fb3e None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.190208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895020b-9ebe-469e-a135-2550acade95b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.212005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.212836 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.213160 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.213445 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.224538 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.225973 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.227126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.227923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.228204 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.228480 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.240151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.241641 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:58.244771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:58.245856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:58.246855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:58.246855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:58.246855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:58.246855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:58.254394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 321/669] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:58.254394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:58.270525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:58.271594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:58.272595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:58.272595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:58.272595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:58.272595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:15:58.272595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 349/670] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:58.272915 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.368273 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c16d0-bb58-4250-b372-8c8d6823744a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.374748 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f207a2-a5d4-4f6b-b5fb-5e8a08016b2e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.402445 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21713eaa-a9ac-426f-9a07-9050dc121ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.403238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713eaa-a9ac-426f-9a07-9050dc121ba3 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.403513 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713eaa-a9ac-426f-9a07-9050dc121ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.403785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21713eaa-a9ac-426f-9a07-9050dc121ba3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.410038 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.410704 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.410985 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.411259 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.412027 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:58.412339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.413853 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.414913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21713eaa-a9ac-426f-9a07-9050dc121ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-21713eaa-a9ac-426f-9a07-9050dc121ba3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:15:58.422986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:58.423986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:15:58.424890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:58.432795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 350/672] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:58.432795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.433193 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 322/672] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:58.434290 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21713eaa-a9ac-426f-9a07-9050dc121ba3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.533465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020a734-6771-44be-8a02-49b3499115e0 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.541776 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21713eaa-a9ac-426f-9a07-9050dc121ba3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.569416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.570940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.571230 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.571504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.572235 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:58.572539 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.573943 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.578636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.579439 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.579724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.580002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.586319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 351/673] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:58.587010 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.587145 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.588599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:58.603119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:58.604135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:58.606071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:58.606071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:58.606071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:58.606071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:58.606071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:58.606071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:58.606071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:58.606071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:58.609182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 323/674] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:58.610217 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.701404 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a8d88-b0d8-43a7-bc61-dfb0a382e007 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.718167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388e151-d218-4ebe-9d09-1a747a31c20f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.742962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.744170 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.744452 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.744884 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.745975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:58.746432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.748094 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.753262 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.754170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.754558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.754950 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.758916 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 352/675] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:58.759887 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.768355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.769504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:15:58.783687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:58.784853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:58.786086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:15:58.786086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:15:58.786086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:15:58.786086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:15:58.786086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:15:58.786086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:15:58.786086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:15:58.786086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:58.792211 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 324/676] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:58.792211 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.890224 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512b7a8-d1aa-477a-8ec6-768aebfa37fc None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.892151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85d0b6-0472-45d2-9776-fd689dffc99e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:58.926727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.927717 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.928252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.928644 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.935077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:58.935926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:58.936377 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:58.937256 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:58.938196 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:58.938679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.941021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.941352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:58.943069 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:58.955577 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 353/677] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:58.956699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:15:58.963531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:58.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:58.965766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:15:58.965766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:15:58.965766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:15:58.965766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:58.970937 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 325/678] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:58.970937 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:59.083390 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9682ec1-2ee0-49cc-8760-b333ab368909 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:59.091495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ce577-b40a-4ae6-a6f2-46dad7be187a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:59.122698 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:59.123470 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.123894 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.124234 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.125503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:59.125883 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:59.128351 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:59.134904 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:59.136249 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.137028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.137807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.139292 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=fb19da929a66435e86ed966bf4a3c9cb) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:59.139836 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:59.146989 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 354/679] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:59.146989 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:59.148796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:59.157852 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 326/680] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:58 2022] GET /identity/v3/role_assignments?user.id=fb19da929a66435e86ed966bf4a3c9cb&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:59.159522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:59.279579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872ab4-468f-47e8-ad3b-7c9a2abd23cb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:59.284615 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a342f-2abd-43ce-be69-fb01c2221552 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:59.318143 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:59.319809 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.320359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.321021 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.322721 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=9006561a3f5340c9b397c4892f7d1f34) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:59.323258 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:59.324922 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:59.326613 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:59.327686 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.328584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.329078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.330290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:59.330771 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:59.332311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:59.337135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 327/681] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:15:59 2022] GET /identity/v3/role_assignments?user.id=9006561a3f5340c9b397c4892f7d1f34&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:59.349592 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 355/682] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:59.358104 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:59.481862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df06bee-b01e-4312-94bc-2769dff2dbe8 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:59.525900 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:59.527045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.527541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.528027 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] PATH_INFO: `/v3/users/swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.541936 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:59.543812 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:15:59.560553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:15:59.561802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:15:59.562899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:15:59.566614 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 328/683] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:15:59.573079 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:59.684471 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3863e-4414-40e5-b89f-9f36dc5fac3c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:59.720558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51deb-6c5c-45ef-a357-82338dc7dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:59.721703 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51deb-6c5c-45ef-a357-82338dc7dd69 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.722202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51deb-6c5c-45ef-a357-82338dc7dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.722742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51deb-6c5c-45ef-a357-82338dc7dd69 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.723874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b51deb-6c5c-45ef-a357-82338dc7dd69 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:15:59.724405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b51deb-6c5c-45ef-a357-82338dc7dd69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:15:59.726182 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b51deb-6c5c-45ef-a357-82338dc7dd69 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:15:59.755309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 356/684] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:15:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:15:59.761573 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:15:59.774950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01438-6cc3-4232-af93-9979d0f6beb2 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.775687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01438-6cc3-4232-af93-9979d0f6beb2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.776240 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01438-6cc3-4232-af93-9979d0f6beb2 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.778110 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 357/685] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:15:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:15:59.783083 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e58daf-8a50-4ee2-88f9-b66b7726f9c3 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.783364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e58daf-8a50-4ee2-88f9-b66b7726f9c3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.783634 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e58daf-8a50-4ee2-88f9-b66b7726f9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.811843 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-56e58daf-8a50-4ee2-88f9-b66b7726f9c3 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:15:59.813033 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-56e58daf-8a50-4ee2-88f9-b66b7726f9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:15:59.880753 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb899df-4fb0-469a-bae4-3db5f06a5d98 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:15:59.924383 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595dc6c-746f-4d94-95e4-6924f9bdd5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:15:59.925060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dc6c-746f-4d94-95e4-6924f9bdd5cd None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:15:59.925343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dc6c-746f-4d94-95e4-6924f9bdd5cd None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:15:59.925613 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dc6c-746f-4d94-95e4-6924f9bdd5cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:15:59.964721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 358/686] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:16:00.000655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e58daf-8a50-4ee2-88f9-b66b7726f9c3 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:00.043835 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595dc6c-746f-4d94-95e4-6924f9bdd5cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:00.045238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595dc6c-746f-4d94-95e4-6924f9bdd5cd None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:00.163137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 329/687] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:15:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:16:00.170609 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595dc6c-746f-4d94-95e4-6924f9bdd5cd None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:00.197994 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e58daf-8a50-4ee2-88f9-b66b7726f9c3 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:00.237068 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffc38c-426a-41ab-8e93-207f0d6d6c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:00.237863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffc38c-426a-41ab-8e93-207f0d6d6c92 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:00.238140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffc38c-426a-41ab-8e93-207f0d6d6c92 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:00.238408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffc38c-426a-41ab-8e93-207f0d6d6c92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:00.266287 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffc38c-426a-41ab-8e93-207f0d6d6c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-86ffc38c-426a-41ab-8e93-207f0d6d6c92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:00.282208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:00.283295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:00.284224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:00.290402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 359/688] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:00.310191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffc38c-426a-41ab-8e93-207f0d6d6c92 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:00.332437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595dc6c-746f-4d94-95e4-6924f9bdd5cd None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:00.375810 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:00.377051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:00.377554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:00.378068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:00.406616 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:00.411289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:00.447073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:00.448125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:00.449142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:00.449142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:00.449142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:00.449142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:00.458567 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 330/689] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:00.463936 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:00.550826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffc38c-426a-41ab-8e93-207f0d6d6c92 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:00.598651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b2421-2a33-47b7-9dd8-5957556a2750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:00.599249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2421-2a33-47b7-9dd8-5957556a2750 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:00.599521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2421-2a33-47b7-9dd8-5957556a2750 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:00.599792 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b2421-2a33-47b7-9dd8-5957556a2750 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:00.600548 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b2421-2a33-47b7-9dd8-5957556a2750 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:00.600853 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b2421-2a33-47b7-9dd8-5957556a2750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:00.603095 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b2421-2a33-47b7-9dd8-5957556a2750 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:00.611223 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 360/690] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:00.612461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68303-ce3b-48da-a12b-27ea353d158e None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:00.612745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68303-ce3b-48da-a12b-27ea353d158e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:00.613022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68303-ce3b-48da-a12b-27ea353d158e None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:00.623015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 361/691] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:00.623015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68303-ce3b-48da-a12b-27ea353d158e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:00.637389 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbff819-2cdb-40ab-b7db-73518d52658c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:00.677939 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184abef-193e-437b-8b36-e6d89353de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:00.678909 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184abef-193e-437b-8b36-e6d89353de52 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:00.679313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184abef-193e-437b-8b36-e6d89353de52 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:00.680626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184abef-193e-437b-8b36-e6d89353de52 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:00.681828 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184abef-193e-437b-8b36-e6d89353de52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:00.682242 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184abef-193e-437b-8b36-e6d89353de52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:00.684142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184abef-193e-437b-8b36-e6d89353de52 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:00.699237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 331/692] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:00.700705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6ac62-6c52-44af-a3bb-0241ee0e8b3f None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:00.701137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6ac62-6c52-44af-a3bb-0241ee0e8b3f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:00.701517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6ac62-6c52-44af-a3bb-0241ee0e8b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:00.722079 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-b6e6ac62-6c52-44af-a3bb-0241ee0e8b3f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:00.723501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-b6e6ac62-6c52-44af-a3bb-0241ee0e8b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:00.746948 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68303-ce3b-48da-a12b-27ea353d158e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:00.793956 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:00.801066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 332/693] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:16:00.801066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6ac62-6c52-44af-a3bb-0241ee0e8b3f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:00.801394 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:00.801885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:00.803716 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:00.825304 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:00.827577 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:00.851610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:00.852596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:00.853623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:00.853623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:00.853623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:00.853623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:00.856370 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 362/694] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:00.857372 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:00.924555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6ac62-6c52-44af-a3bb-0241ee0e8b3f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:00.959668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:00.960754 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:00.961267 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:00.961697 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:00.971260 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:00.972764 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:00.977329 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2102020-ac49-49e5-bc1c-a58fd7391cd0 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:00.986541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:00.987790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:00.988994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:00.988994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:00.988994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:00.988994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:00.988994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:00.988994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:00.988994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:00.988994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:00.994829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 333/695] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:00.995661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:01.018947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.020480 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.021283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.021803 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.040062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.042533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:01.065375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:01.066360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:01.067356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:01.067356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:01.067356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:01.067356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:01.075807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 363/696] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:01.076847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:01.104791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a33fb5-036f-435c-8de3-4c7152f4c49e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:01.145249 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.146522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.147164 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.147588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.149495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:01.150093 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.152387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.167723 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 334/697] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:01.168772 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:01.205584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809e50e-a180-44f7-906d-e8e8f3a61fd5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:01.258687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.259563 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.260040 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.260479 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.261468 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:01.261994 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.263638 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.280036 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 364/698] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:01.281160 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:01.289259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421e9e6-0799-4552-a456-07ff054d8df9 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:01.324871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.325677 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.326161 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.326510 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.328192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:01.328574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.330052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.344666 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 335/699] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:01.346054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:01.399171 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71350705-70a6-4228-bb7d-5fe1548baf99 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:01.433343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.434288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.434566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.434920 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.442580 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.444050 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.453203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0038f8-a2f2-40e9-85f5-8e144fb675dd None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:01.459380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:01.460412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:01.461526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:01.465901 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 365/700] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:16:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:01.467044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:01.492815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.494336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.494835 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.495304 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.496471 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=e81aea4ac1ae45e2997c6c70e8c0028a) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:01.497022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.498782 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.506189 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 336/701] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:01 2022] GET /identity/v3/role_assignments?user.id=e81aea4ac1ae45e2997c6c70e8c0028a&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:01.507333 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:01.615971 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9fb50-aebb-4a13-bc44-876898e22c60 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:01.625610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682a022-823c-48df-a7e2-bc4ab5359140 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:01.653263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.654261 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.654762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.655233 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.656329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:01.656869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.658508 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.660985 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fc293-7346-4bbd-84e1-23b7da2dccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.662129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc293-7346-4bbd-84e1-23b7da2dccdb None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.662129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc293-7346-4bbd-84e1-23b7da2dccdb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.662361 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc293-7346-4bbd-84e1-23b7da2dccdb None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.663114 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fc293-7346-4bbd-84e1-23b7da2dccdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.664697 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fc293-7346-4bbd-84e1-23b7da2dccdb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.680452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 337/702] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:16:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:01.690864 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:01.692967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 366/703] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:16:01 2022] POST /identity/v3/users => generated 254 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:01.812669 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf04b4-7840-41ad-ae22-f6d9f056a070 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:01.860115 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0af3ee-354a-4340-914b-9be39c5040e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:01.860789 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af3ee-354a-4340-914b-9be39c5040e7 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.861129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af3ee-354a-4340-914b-9be39c5040e7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.861413 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af3ee-354a-4340-914b-9be39c5040e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.940286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0af3ee-354a-4340-914b-9be39c5040e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:01.941705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0af3ee-354a-4340-914b-9be39c5040e7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:01.968550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c72e31-d997-403b-8659-bad486f7d2bb None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.968893 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c72e31-d997-403b-8659-bad486f7d2bb None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.969140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c72e31-d997-403b-8659-bad486f7d2bb None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.970739 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 367/704] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:01.976082 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98b058-607c-48c5-a2a4-5daa0375a5ff None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:01.976395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98b058-607c-48c5-a2a4-5daa0375a5ff None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:01.976666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98b058-607c-48c5-a2a4-5daa0375a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:01.995867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-aa98b058-607c-48c5-a2a4-5daa0375a5ff None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:01.997052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-aa98b058-607c-48c5-a2a4-5daa0375a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:02.054439 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 338/705] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:16:02.056445 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d178da-1797-4146-9c78-50a4c5aac3be None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.056570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d178da-1797-4146-9c78-50a4c5aac3be None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.056816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d178da-1797-4146-9c78-50a4c5aac3be None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.058552 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 339/706] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:02.061929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d178da-1797-4146-9c78-50a4c5aac3be None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.082948 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 368/707] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:16:02.084272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be622f-d628-408b-a9a3-416c70aa348f None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.084562 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be622f-d628-408b-a9a3-416c70aa348f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.084870 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be622f-d628-408b-a9a3-416c70aa348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.108089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-d8be622f-d628-408b-a9a3-416c70aa348f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:02.109505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-d8be622f-d628-408b-a9a3-416c70aa348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:02.185991 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d178da-1797-4146-9c78-50a4c5aac3be None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.197402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 369/708] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:16:02.198271 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be622f-d628-408b-a9a3-416c70aa348f None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.227455 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.228637 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.229198 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.229749 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.245911 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:02.247449 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:02.271619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:02.272659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:02.274790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:02.274790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:02.274790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:02.274790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:02.274790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 340/709] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:02.274790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.329487 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be622f-d628-408b-a9a3-416c70aa348f None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.373865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.374968 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.375506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.375980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.377161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:02.377658 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eeaf40-a61b-4aee-9df4-6a33e3ac0227 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.378570 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:02.398801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:02.400699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:02.401904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:02.405574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 370/710] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:02.406471 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.420402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.421601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.422053 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.422477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] PATH_INFO: `/v3/services/network` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.423289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:02.424870 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:02.443801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:02.444954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:02.446478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:02.448151 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 341/711] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:02.448151 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.529369 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbf3b8-d617-4cb1-b478-1669ba3b6647 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.577079 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.577771 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.578050 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.578323 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.579117 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:02.579424 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:02.580862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:02.583841 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad65e73-6def-4464-828c-34cc72a92826 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.597803 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 371/712] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:02.598709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.629708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.631328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.631841 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.632334 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.633512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:02.634825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:02.636425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:02.642564 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 342/713] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:02.643993 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.701577 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a9bfa-f766-4d8d-a7b5-f4c38d6be42b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.741074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.743258 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.744022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.744489 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.746014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] RBAC: Adding query filter params (name=network) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:02.746535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:02.748529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:02.754303 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 372/714] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:02.755576 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.768291 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab21b1e-518f-4ba5-939d-7649c9724489 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.803020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.803819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.804103 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.804467 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.811703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:02.813167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:02.827374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:02.828359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:02.829372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:02.829372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:02.829372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:02.829372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:02.829372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:02.829372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:02.829372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:02.829372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:02.833433 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 343/715] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:02.834097 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.895445 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f554e7-b0df-4a19-9dee-9b899a54b9ae None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.942893 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.943559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.943885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.944163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.944880 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:02.945255 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:02.946690 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:02.952484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 373/716] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:02.953312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:02.958219 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473f5d4-a9f9-483d-a5ca-8a7757dda78c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:02.996377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:02.997065 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:02.997361 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:02.997641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:02.998397 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] RBAC: Adding query filter params (type=network) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:02.998721 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:03.000453 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:03.006850 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 344/717] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:03.206475 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7bd4-80a9-4b79-9a1b-a466b60c201a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:03.250257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465c9b1-b977-4548-b8ba-4631950ffc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:03.251371 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465c9b1-b977-4548-b8ba-4631950ffc49 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:03.251874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465c9b1-b977-4548-b8ba-4631950ffc49 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:03.252363 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465c9b1-b977-4548-b8ba-4631950ffc49 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:03.253416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465c9b1-b977-4548-b8ba-4631950ffc49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:03.254500 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465c9b1-b977-4548-b8ba-4631950ffc49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:03.256205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465c9b1-b977-4548-b8ba-4631950ffc49 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:03.269328 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 374/718] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:03.276227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:03.391633 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a248a9-0f5a-4851-8ed4-69a9156f5fed None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:03.432170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae973b-aead-49f6-bf74-b95466532a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:03.432833 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae973b-aead-49f6-bf74-b95466532a14 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:03.433147 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae973b-aead-49f6-bf74-b95466532a14 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:03.433422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae973b-aead-49f6-bf74-b95466532a14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:03.434251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae973b-aead-49f6-bf74-b95466532a14 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=f3a73bc43c23400d8ee7d4e880aa7242) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:03.434566 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae973b-aead-49f6-bf74-b95466532a14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:03.435985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae973b-aead-49f6-bf74-b95466532a14 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:03.441868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 345/719] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:03 2022] GET /identity/v3/role_assignments?user.id=f3a73bc43c23400d8ee7d4e880aa7242&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:04.497016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec00ccc-f92b-40e3-b946-a3c1ce064520 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:04.497589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec00ccc-f92b-40e3-b946-a3c1ce064520 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:04.497589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec00ccc-f92b-40e3-b946-a3c1ce064520 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:04.499293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 375/720] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:04.500572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1722-2f45-460b-9c46-d7f43a43de32 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:04.501085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1722-2f45-460b-9c46-d7f43a43de32 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:04.501251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1722-2f45-460b-9c46-d7f43a43de32 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:04.502741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 346/721] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:04.504525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9563e-cc07-4fae-8084-3e4c80518c4b None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:04.504815 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9563e-cc07-4fae-8084-3e4c80518c4b None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:04.505126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9563e-cc07-4fae-8084-3e4c80518c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:04.507822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9a16-cea0-4903-83f6-e1b74974228e None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:04.508053 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9a16-cea0-4903-83f6-e1b74974228e None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:04.508341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9a16-cea0-4903-83f6-e1b74974228e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:04.529057 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-92cb9a16-cea0-4903-83f6-e1b74974228e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:04.530268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-92cb9a16-cea0-4903-83f6-e1b74974228e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:04.546068 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-71a9563e-cc07-4fae-8084-3e4c80518c4b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:04.547279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-71a9563e-cc07-4fae-8084-3e4c80518c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:04.628479 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 347/722] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:16:04.641517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 376/723] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:16:04.664526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb9a16-cea0-4903-83f6-e1b74974228e None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:04.672999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9563e-cc07-4fae-8084-3e4c80518c4b None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:04.826991 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb9a16-cea0-4903-83f6-e1b74974228e None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:04.847478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9563e-cc07-4fae-8084-3e4c80518c4b None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:04.892562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a0fac-661b-4172-8eef-80123f989556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:04.893387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0fac-661b-4172-8eef-80123f989556 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:04.893673 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0fac-661b-4172-8eef-80123f989556 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:04.893952 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a0fac-661b-4172-8eef-80123f989556 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:04.911663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a0fac-661b-4172-8eef-80123f989556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:04.918049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab7fcd-67ec-42f2-b58c-70e3ed472650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:04.919034 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab7fcd-67ec-42f2-b58c-70e3ed472650 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:04.919306 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab7fcd-67ec-42f2-b58c-70e3ed472650 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:04.919588 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab7fcd-67ec-42f2-b58c-70e3ed472650 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-396a0fac-661b-4172-8eef-80123f989556 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:04.924626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:04.925831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:04.927288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 348/724] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:16:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:04.939179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a0fac-661b-4172-8eef-80123f989556 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:04.944456 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab7fcd-67ec-42f2-b58c-70e3ed472650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-27ab7fcd-67ec-42f2-b58c-70e3ed472650 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:04.952280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:04.954293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:04.955486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 377/725] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:04.960044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab7fcd-67ec-42f2-b58c-70e3ed472650 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:05.087325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a0fac-661b-4172-8eef-80123f989556 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:05.113096 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab7fcd-67ec-42f2-b58c-70e3ed472650 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:05.122332 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:05.123006 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.123293 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.123574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.124335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] RBAC: Adding query filter params (name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:05.124710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:05.126120 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:05.134809 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 349/726] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:16:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:05.139814 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:05.181765 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:05.182454 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.182738 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.183013 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.183775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:05.184080 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:05.185488 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:05.204529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 378/727] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:05.209494 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:05.249948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1a597-4d30-4d07-9589-6776efa89c4f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:05.287397 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:05.288221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.288511 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.288846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.300861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:05.302501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:05.322801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:05.323844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:05.324815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:05.324815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:05.324815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:05.324815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:05.324815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 350/728] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:05.333989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:05.381806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096416c-ed72-4214-adf3-79842e4c6d1c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:05.415031 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:05.415827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.416171 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.416429 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.426821 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:05.428268 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:05.446436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:05.447442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:05.448435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:05.448435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:05.448435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:05.448435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:05.452196 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 379/729] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:05.453405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:05.553994 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225c46-aa86-4054-8a79-375538b30fe8 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:05.556501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7301d52-cd23-4bd4-af26-aaaa31aed598 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:05.590977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:05.591643 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.591926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.592191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.592910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:05.593244 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:05.594594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:05.611294 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 380/730] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:05.612645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:05.630996 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3109889-f487-406d-962b-ebc260687efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:05.631686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3109889-f487-406d-962b-ebc260687efa None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.632021 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3109889-f487-406d-962b-ebc260687efa None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.632240 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3109889-f487-406d-962b-ebc260687efa None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.633565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3109889-f487-406d-962b-ebc260687efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:05.633873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3109889-f487-406d-962b-ebc260687efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:05.635268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3109889-f487-406d-962b-ebc260687efa None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:05.670124 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 351/731] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:05.675657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3109889-f487-406d-962b-ebc260687efa None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:05.777471 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff74783-8b56-4388-99d5-52119faeb78f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:05.814815 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cfd74-c5c5-4197-9c53-d409946feba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:05.815611 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cfd74-c5c5-4197-9c53-d409946feba1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.815890 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cfd74-c5c5-4197-9c53-d409946feba1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.816162 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cfd74-c5c5-4197-9c53-d409946feba1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.822710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3109889-f487-406d-962b-ebc260687efa None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:05.823518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cfd74-c5c5-4197-9c53-d409946feba1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:05.825006 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cfd74-c5c5-4197-9c53-d409946feba1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-454cfd74-c5c5-4197-9c53-d409946feba1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:05.839648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:05.841586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 381/732] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:05.842640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32565207-fa8b-4f72-bdd7-749e21bd2745 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.843376 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32565207-fa8b-4f72-bdd7-749e21bd2745 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.843376 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32565207-fa8b-4f72-bdd7-749e21bd2745 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.847032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 382/733] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:05.847032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160974a-d252-481c-8e84-2dbf27de1aff None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.847032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160974a-d252-481c-8e84-2dbf27de1aff None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.847032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160974a-d252-481c-8e84-2dbf27de1aff None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.847651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 383/734] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:05.850472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207340-beeb-45ba-bee8-48a5b4272b12 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.850752 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207340-beeb-45ba-bee8-48a5b4272b12 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.851018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207340-beeb-45ba-bee8-48a5b4272b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.866033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:05.867754 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.867754 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.867966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.873713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-6d207340-beeb-45ba-bee8-48a5b4272b12 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:05.874882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-6d207340-beeb-45ba-bee8-48a5b4272b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:05.875445 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:05.876888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:05.891706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:05.892727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 352/735] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:16:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:05.898740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:05.953947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 384/736] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:16:05.955247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4aad3-267a-443a-b85a-485f909ae70a None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:05.955537 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4aad3-267a-443a-b85a-485f909ae70a None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:05.955812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4aad3-267a-443a-b85a-485f909ae70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:05.975674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-89e4aad3-267a-443a-b85a-485f909ae70a None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:05.977752 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-89e4aad3-267a-443a-b85a-485f909ae70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:06.000691 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32b277-d628-45a9-814c-65504db61e3a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:06.034001 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:06.034787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.035071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.035347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.036213 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:06.036531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:06.038084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:06.048899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 385/737] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:16:06.050175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4aad3-267a-443a-b85a-485f909ae70a None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:06.067591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 353/738] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:16:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:06.068438 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:06.166712 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f18f2-ff20-4436-837b-acbf60ac3448 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:06.168756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4aad3-267a-443a-b85a-485f909ae70a None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:06.200481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae0837-eb25-4dd7-99b9-70987a27999b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:06.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0837-eb25-4dd7-99b9-70987a27999b None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.201463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0837-eb25-4dd7-99b9-70987a27999b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.201581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:06.201762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0837-eb25-4dd7-99b9-70987a27999b None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.202201 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.202459 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae0837-eb25-4dd7-99b9-70987a27999b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:06.202561 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.202737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.203567 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:06.203819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae0837-eb25-4dd7-99b9-70987a27999b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:06.203948 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:06.205288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:06.229075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 354/739] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:16:06 2022] POST /identity/v3/services => generated 240 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:06.229075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae0837-eb25-4dd7-99b9-70987a27999b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:06.235857 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 386/740] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:16:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:06.236987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:06.340323 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae0837-eb25-4dd7-99b9-70987a27999b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:06.345693 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c468b1-94b1-4952-9c59-3742fbd91d56 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:06.381656 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b3dbd-8541-4cf1-bd32-cb3c1856023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:06.381945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e769413-087e-46d9-9a5b-d7115dc76c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:06.382336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b3dbd-8541-4cf1-bd32-cb3c1856023c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.382659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b3dbd-8541-4cf1-bd32-cb3c1856023c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.382773 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e769413-087e-46d9-9a5b-d7115dc76c6e None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.382949 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e769413-087e-46d9-9a5b-d7115dc76c6e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.383061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b3dbd-8541-4cf1-bd32-cb3c1856023c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.383223 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e769413-087e-46d9-9a5b-d7115dc76c6e None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.383931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e769413-087e-46d9-9a5b-d7115dc76c6e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:06.385406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e769413-087e-46d9-9a5b-d7115dc76c6e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:06.401228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 387/741] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:16:06 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:06.402560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3393ef2-24b1-4aff-950b-af736d18bc89 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.402847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3393ef2-24b1-4aff-950b-af736d18bc89 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.403134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3393ef2-24b1-4aff-950b-af736d18bc89 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.404393 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 388/742] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:06.408828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3393ef2-24b1-4aff-950b-af736d18bc89 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:06.473221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b3dbd-8541-4cf1-bd32-cb3c1856023c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:06.474983 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b3dbd-8541-4cf1-bd32-cb3c1856023c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:06.558384 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3393ef2-24b1-4aff-950b-af736d18bc89 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:06.591127 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 355/743] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:06 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:16:06.592527 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1a52e-5b17-4954-99b4-173d188fa72f None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.592825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1a52e-5b17-4954-99b4-173d188fa72f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.593130 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1a52e-5b17-4954-99b4-173d188fa72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.597468 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dce8f-3afb-4ca1-85c4-68e6fa2301e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:06.598124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dce8f-3afb-4ca1-85c4-68e6fa2301e9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.598400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dce8f-3afb-4ca1-85c4-68e6fa2301e9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.598668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dce8f-3afb-4ca1-85c4-68e6fa2301e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.610807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-bfe1a52e-5b17-4954-99b4-173d188fa72f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:06.612313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-bfe1a52e-5b17-4954-99b4-173d188fa72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:06.672893 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dce8f-3afb-4ca1-85c4-68e6fa2301e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:06.674435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dce8f-3afb-4ca1-85c4-68e6fa2301e9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:06.691120 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 356/744] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:06.691979 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1a52e-5b17-4954-99b4-173d188fa72f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:06.777233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 389/745] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:16:06.778071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dce8f-3afb-4ca1-85c4-68e6fa2301e9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:06.829767 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1a52e-5b17-4954-99b4-173d188fa72f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:06.865460 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:06.866262 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.866546 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.866857 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.876617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dce8f-3afb-4ca1-85c4-68e6fa2301e9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:06.877727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:06.879183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:06.897072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:06.898628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:06.899601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:06.899601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:06.899601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:06.899601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:06.899601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 357/746] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:06.901319 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:06.910741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf624c8-bc00-4053-8917-21f8811366cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:06.911533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf624c8-bc00-4053-8917-21f8811366cc None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:06.911810 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf624c8-bc00-4053-8917-21f8811366cc None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:06.912078 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf624c8-bc00-4053-8917-21f8811366cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:06.923446 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf624c8-bc00-4053-8917-21f8811366cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-8bf624c8-bc00-4053-8917-21f8811366cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:06.931684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:06.933976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 390/747] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:06.936167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf624c8-bc00-4053-8917-21f8811366cc None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.015477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fb50c-7c04-4d2a-982c-90bc8e2497e1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.033994 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf624c8-bc00-4053-8917-21f8811366cc None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.056038 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.057296 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.057843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.058738 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.072173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.073908 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.074025 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.074618 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.074876 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.075217 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.075947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:07.076245 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.077828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.096635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 391/748] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:07.096635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:07.102167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:07.103132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:07.104114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:07.104114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:07.104114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:07.104114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:07.111282 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 358/749] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:07.111282 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.219635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33bfff-2a27-462c-b503-86f5b51fbbb3 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.221512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3184f03-5005-4587-98a6-31a1d7290ce9 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.268157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.268676 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.268972 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.269370 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.269722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.270312 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.270608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:07.270960 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.271068 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.271523 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.272784 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.282811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.284556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.287733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 392/750] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:07.289084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:07.302519 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:07.303515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:07.304624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:07.304624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:07.304624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:07.304624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:07.304624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:07.304624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:07.304624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:07.304624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:07.311714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 359/751] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:07.311714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.416052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b8c02-9a6a-4ff7-8c54-533241c7091d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.456737 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.457919 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.458410 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.458909 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.459865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:07.460403 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.462411 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.480596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 393/752] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:07.481993 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.561101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94d67b-7b0b-46f4-ad8c-d2e4e473a2e7 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.586635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a16027-e021-469a-8edb-d430dd538390 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.597193 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.597969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.598258 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.598534 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.609282 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.610741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.625457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.626429 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.626918 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.627414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.628430 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:07.628951 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:07.629757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:07.630719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:07.631684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:07.631684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:07.631684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:07.631684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:07.633927 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 360/753] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:07.634149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.634821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.652565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 394/754] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:07.654018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.740924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbea347-9dcd-4b3d-be91-7b17f24425a8 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.753310 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c60e9-bca9-4115-a493-9e082dfd4366 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.776704 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.777294 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.777582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.777860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.784747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.786707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.788431 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.789413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.789966 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.790444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.791408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:07.791927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.793510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:07.800975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:07.801959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:07.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:07.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:07.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:07.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:07.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:07.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:07.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:07.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:07.807325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 361/755] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:07.808178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.809977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 395/756] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:07.811043 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.912885 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c427b0-e683-4ea2-bf50-6ae2efe44319 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.923369 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b23ab2-01a1-4c5f-a491-eeeb340113ed None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:07.950095 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.951175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.951711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.952220 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.953260 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:07.953797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.955418 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.959687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:07.960353 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:07.960668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:07.960947 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:07.961825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=e81aea4ac1ae45e2997c6c70e8c0028a) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:07.962142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:07.963884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:07.969607 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 396/757] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:07.969795 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 362/758] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/role_assignments?user.id=e81aea4ac1ae45e2997c6c70e8c0028a&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:07.970489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:07.977825 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:08.086710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88c86-8691-4738-b610-b119371f0a69 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:08.088530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d4974-32dd-4256-8541-7e4e00ec9cbc None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:08.124911 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:08.125758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:08.126051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:08.126331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:08.128345 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:08.129311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:08.129819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:08.130309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:08.131382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=e28363f2360842958bfe92394afe9651) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:08.131925 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:08.133681 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:08.135122 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:08.136605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:08.141098 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 397/759] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/role_assignments?user.id=e28363f2360842958bfe92394afe9651&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:08.160543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:08.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:08.162729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:08.166550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 363/760] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:16:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:08.171904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:08.338816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690873-d462-41b0-8093-4429b54d58f1 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:08.394730 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:08.395420 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:08.395700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:08.395972 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:08.396819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:08.397161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:08.399206 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:08.423484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 398/761] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:16:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:08.429438 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:08.582883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80629d-9aa9-4570-850d-bc0b7903c020 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:08.634685 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956fa34-9440-4cb4-bc1e-1f17286e7d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:08.635486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956fa34-9440-4cb4-bc1e-1f17286e7d1a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:08.635760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956fa34-9440-4cb4-bc1e-1f17286e7d1a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:08.636031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956fa34-9440-4cb4-bc1e-1f17286e7d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:08.650664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956fa34-9440-4cb4-bc1e-1f17286e7d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:08.652101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956fa34-9440-4cb4-bc1e-1f17286e7d1a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-9956fa34-9440-4cb4-bc1e-1f17286e7d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:08.672521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:08.673629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:08.674619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:08.674619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:08.674619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:08.674619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:08.677152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 364/762] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:08.679346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:08.760741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128ac1-2d40-4dab-a190-dbf735e54f69 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:08.761381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128ac1-2d40-4dab-a190-dbf735e54f69 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:08.761729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128ac1-2d40-4dab-a190-dbf735e54f69 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:08.763256 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 365/763] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:08.771523 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336bb06-8187-433d-b33c-d292969d3d60 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:08.771919 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336bb06-8187-433d-b33c-d292969d3d60 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:08.772297 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336bb06-8187-433d-b33c-d292969d3d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:08.787396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c2ee3-5914-4d8e-99a0-2fe96a843c4f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:08.802962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-1336bb06-8187-433d-b33c-d292969d3d60 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:08.804146 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-1336bb06-8187-433d-b33c-d292969d3d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:08.880220 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:08.881738 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 366/764] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:08.881877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:08.881877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:08.881877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:08.881877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:08.881877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:08.881877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:08.894444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 399/765] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:08.900308 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336bb06-8187-433d-b33c-d292969d3d60 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:08.909210 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.098350 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fafe7b-015d-4d6f-8523-f49415dc33ff None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.114401 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336bb06-8187-433d-b33c-d292969d3d60 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.152950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79ea2d-d820-4e39-9207-57cd6645749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.153780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79ea2d-d820-4e39-9207-57cd6645749b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.154060 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79ea2d-d820-4e39-9207-57cd6645749b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.154392 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79ea2d-d820-4e39-9207-57cd6645749b None admin] PATH_INFO: `/v3/services/network` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.155049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79ea2d-d820-4e39-9207-57cd6645749b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.156408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79ea2d-d820-4e39-9207-57cd6645749b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.159421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125b32c-d97b-4ad3-a190-a70b8e31d098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.160681 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125b32c-d97b-4ad3-a190-a70b8e31d098 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.161165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125b32c-d97b-4ad3-a190-a70b8e31d098 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.161591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125b32c-d97b-4ad3-a190-a70b8e31d098 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-8f79ea2d-d820-4e39-9207-57cd6645749b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:09.167974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:09.169283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:09.170218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:09.173376 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 400/766] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:08 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:09.174869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125b32c-d97b-4ad3-a190-a70b8e31d098 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.175514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226aff8-0bc7-4d69-ad20-6c3cae6dcf77 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.176008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226aff8-0bc7-4d69-ad20-6c3cae6dcf77 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.176413 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125b32c-d97b-4ad3-a190-a70b8e31d098 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.176719 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226aff8-0bc7-4d69-ad20-6c3cae6dcf77 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.178751 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 401/767] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:09.180535 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226aff8-0bc7-4d69-ad20-6c3cae6dcf77 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e125b32c-d97b-4ad3-a190-a70b8e31d098 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:09.194735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:09.196331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:09.197343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:09.197343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:09.197343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:09.197343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:09.197343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:09.197343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:09.197343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:09.197343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:09.201205 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 367/768] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:09.202452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb78d96-3357-4ac9-815c-1520bfdc09f4 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.202740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb78d96-3357-4ac9-815c-1520bfdc09f4 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.203019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb78d96-3357-4ac9-815c-1520bfdc09f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.222368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-ddb78d96-3357-4ac9-815c-1520bfdc09f4 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:09.224259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-ddb78d96-3357-4ac9-815c-1520bfdc09f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:09.307169 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 368/769] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:09.308483 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb78d96-3357-4ac9-815c-1520bfdc09f4 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.325388 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226aff8-0bc7-4d69-ad20-6c3cae6dcf77 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.365511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.366538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.367622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.368119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.369377 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] RBAC: Adding query filter params (name=network) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:09.369906 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.371481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.376694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 402/770] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:09.378012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.434361 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb78d96-3357-4ac9-815c-1520bfdc09f4 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.470683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.471585 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.471706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.471892 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.472664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:09.473077 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.474417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.492843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 369/771] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:09.493868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.499263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e04b99-27bb-44a6-aabf-2710a91f1898 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.542226 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.543683 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.544204 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.544709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.546077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.548146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:09.565581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:09.566720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:09.567636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:09.570632 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 403/772] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:16:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:09.576010 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.607201 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a7d64-f54e-4f28-ae94-bc1b6df3c11d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.650882 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.652632 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.653181 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.653966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.655165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] RBAC: Adding query filter params (type=network) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:09.655684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.657921 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.664739 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 370/773] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:09.665848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.676914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f0cfd-784d-485e-89c0-cb6bd9f66c93 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.710032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e76be8-8ca6-4d14-a2c3-b556ff4f6970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.711573 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e76be8-8ca6-4d14-a2c3-b556ff4f6970 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.711918 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e76be8-8ca6-4d14-a2c3-b556ff4f6970 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.712265 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e76be8-8ca6-4d14-a2c3-b556ff4f6970 None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/f3a73bc43c23400d8ee7d4e880aa7242/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.714323 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e76be8-8ca6-4d14-a2c3-b556ff4f6970 None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=f3a73bc43c23400d8ee7d4e880aa7242, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.715912 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e76be8-8ca6-4d14-a2c3-b556ff4f6970 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.735116 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 404/774] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:16:09 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/f3a73bc43c23400d8ee7d4e880aa7242/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:16:09.736002 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e76be8-8ca6-4d14-a2c3-b556ff4f6970 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.793642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3c85d-e5f2-4629-8379-2c6142127579 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.839304 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.840502 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.841279 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.841978 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.842898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:09.843441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.843787 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e76be8-8ca6-4d14-a2c3-b556ff4f6970 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:09.845382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.852989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 371/775] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:16:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:09.859670 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:09.899968 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:09.903525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:09.903525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:09.904455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:09.907927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] RBAC: Adding query filter params (interface=public, service_id=6a36eda6458144f0b20f9db94cadd713, region_id=RegionOne) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:09.908250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:09.909695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:09.927531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 405/776] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:16:09 2022] GET /identity/v3/endpoints?service_id=6a36eda6458144f0b20f9db94cadd713&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:09.933277 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:10.043134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd96be7-71d4-4338-8ff6-fa49e5b3f032 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:10.077606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e50c3-86d3-4f66-b032-ae9cd228dc17 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:10.080302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:10.081285 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.081787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.082284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.083234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:10.083755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:10.086277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:10.091936 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 372/777] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:16:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:10.097859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:10.115825 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc2db-0210-4576-9190-c89d08649cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:10.116838 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc2db-0210-4576-9190-c89d08649cb2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.117218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc2db-0210-4576-9190-c89d08649cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.117498 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc2db-0210-4576-9190-c89d08649cb2 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.118205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbc2db-0210-4576-9190-c89d08649cb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:10.120331 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbc2db-0210-4576-9190-c89d08649cb2 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:10.128774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 406/778] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:16:09 2022] GET /identity/v3/services => generated 772 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:10.214512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2303cda-d1aa-4dd0-abab-2db5d9ac1964 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:10.276731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343db8b4-468a-492f-9412-ae3c19b15976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:10.277461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343db8b4-468a-492f-9412-ae3c19b15976 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.277750 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343db8b4-468a-492f-9412-ae3c19b15976 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.278026 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343db8b4-468a-492f-9412-ae3c19b15976 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.278835 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343db8b4-468a-492f-9412-ae3c19b15976 None admin] RBAC: Adding query filter params (interface=public, service_id=59b87c1d137d428f84be78ba56998623, region_id=RegionOne) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:10.279145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343db8b4-468a-492f-9412-ae3c19b15976 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:10.280528 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343db8b4-468a-492f-9412-ae3c19b15976 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:10.295166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 373/779] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:16:10 2022] GET /identity/v3/endpoints?service_id=59b87c1d137d428f84be78ba56998623&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:10.300870 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc2db-0210-4576-9190-c89d08649cb2 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:10.421120 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc2db-0210-4576-9190-c89d08649cb2 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:10.459480 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050808ba-a9c1-4567-b85a-92308e4c96c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:10.460155 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050808ba-a9c1-4567-b85a-92308e4c96c4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.460475 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050808ba-a9c1-4567-b85a-92308e4c96c4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.460756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050808ba-a9c1-4567-b85a-92308e4c96c4 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.461483 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050808ba-a9c1-4567-b85a-92308e4c96c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:10.462852 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050808ba-a9c1-4567-b85a-92308e4c96c4 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:10.468465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 407/780] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:16:10 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:10.576028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc5abf-973e-4d8a-a4bb-e2abd69fa975 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.576417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc5abf-973e-4d8a-a4bb-e2abd69fa975 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.576664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc5abf-973e-4d8a-a4bb-e2abd69fa975 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.578345 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 374/781] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:10.584343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39f622-7127-437f-9fb3-f505f350fa9c None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.584649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39f622-7127-437f-9fb3-f505f350fa9c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.584900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39f622-7127-437f-9fb3-f505f350fa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.610158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-fe39f622-7127-437f-9fb3-f505f350fa9c None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:10.611390 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-fe39f622-7127-437f-9fb3-f505f350fa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:10.708530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 408/782] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:10.733194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc5abf-973e-4d8a-a4bb-e2abd69fa975 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:10.842785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc5abf-973e-4d8a-a4bb-e2abd69fa975 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:10.854945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb4f9-01f4-478f-a522-9f69743a3b9c None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.855484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb4f9-01f4-478f-a522-9f69743a3b9c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.855484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb4f9-01f4-478f-a522-9f69743a3b9c None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.857005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 409/783] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:10.864119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692f51b-94e6-4894-a0cb-ab6df4027d85 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.864119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692f51b-94e6-4894-a0cb-ab6df4027d85 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.864119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692f51b-94e6-4894-a0cb-ab6df4027d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.888512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8d2ec-2f05-4f7e-829d-78777574f2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:10.889350 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8d2ec-2f05-4f7e-829d-78777574f2bc None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:10.889638 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8d2ec-2f05-4f7e-829d-78777574f2bc None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:10.889914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8d2ec-2f05-4f7e-829d-78777574f2bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:10.890684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-9692f51b-94e6-4894-a0cb-ab6df4027d85 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:10.892297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-9692f51b-94e6-4894-a0cb-ab6df4027d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:10.906089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8d2ec-2f05-4f7e-829d-78777574f2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e0e8d2ec-2f05-4f7e-829d-78777574f2bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:10.919881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:10.921019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:10.922501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 375/784] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:10.930216 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8d2ec-2f05-4f7e-829d-78777574f2bc None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:10.981084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 410/785] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:11.001630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692f51b-94e6-4894-a0cb-ab6df4027d85 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:11.084569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8d2ec-2f05-4f7e-829d-78777574f2bc None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:11.102488 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692f51b-94e6-4894-a0cb-ab6df4027d85 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:11.138065 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f341-662c-460d-8258-884244803c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:11.138874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f341-662c-460d-8258-884244803c4c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:11.139159 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f341-662c-460d-8258-884244803c4c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:11.139436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f341-662c-460d-8258-884244803c4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:11.150554 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4f341-662c-460d-8258-884244803c4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:11.158644 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:11.159323 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:11.159605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:11.159879 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-7fa4f341-662c-460d-8258-884244803c4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:11.160197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:11.161233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:16:11.162165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:11.162861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:11.162861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:11.163012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 411/786] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:11.163371 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:11.170617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f341-662c-460d-8258-884244803c4c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:11.176384 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 376/787] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:11.187708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:11.310448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f341-662c-460d-8258-884244803c4c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:11.377331 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:11.378003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:11.378281 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:11.378553 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:11.379371 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:11.379678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:11.383628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:11.400436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 412/788] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:11.406546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:11.412471 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b0b14-ce73-4056-a6ef-01d86238f3c1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:11.472338 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:11.473183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:11.473463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:11.473731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:11.489980 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:11.491470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:11.522421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:11.523468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:11.524456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:11.524456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:11.524456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:11.524456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:11.525967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff8433-c7c3-45ab-be6d-156d86dc02e5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:11.529551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 377/789] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:11.535041 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:11.562815 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:11.563603 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:11.563881 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:11.564154 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:11.576998 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:11.578485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:11.602655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:11.603731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:11.604712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:11.604712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:11.604712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:11.604712 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:11.613061 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 413/790] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:11.613061 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:11.710725 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc72573-f471-463a-85f2-56895fb784ec None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:11.738550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162c33c-0228-49a1-8cf6-5a94e951381b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:11.750837 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:11.751542 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:11.751825 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:11.752096 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:11.752818 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:11.753141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:11.754503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:11.772402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 414/791] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:11.773903 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:11.845754 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:11.846440 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:11.846728 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:11.847014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:11.847749 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:11.848058 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:11.849475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:11.876032 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 378/792] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:11.881653 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:11.884546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd147-abd0-473c-8481-d239f94a939a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:11.923272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:11.924081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:11.924356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:11.924624 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:11.933731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:11.935372 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:11.952426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:11.954808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:11.957233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:11.965298 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 415/793] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:11.967518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:12.052870 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc476d89-4faf-4ad6-a57e-1f11bf2f3611 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:12.094441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:12.095253 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.095531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.095807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.108612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:12.110388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:12.137380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:12.138601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:12.139642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 379/794] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:12.146476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:12.148616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cb901-c273-4309-bc99-84ad1fb0a798 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:12.185090 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:12.185907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.186233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.186508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.187512 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:12.187821 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:12.189321 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:12.212371 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 416/795] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:16:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:12.214937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:12.321052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300236da-aa3d-4e52-8686-13f51d75f3cb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:12.384460 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035940a-f163-4443-a78d-d4b1db9498a3 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:12.387748 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:12.388541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.388813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.389096 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.401895 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:12.403350 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:12.422148 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:12.422822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.423108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.423380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.424221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:12.424524 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:12.425974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:12.439163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:12.441442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:12.443708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:12.443708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:12.443708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:12.443708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:12.443708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 417/796] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:12.449087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 380/797] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:16:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:12.450091 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:12.454747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:12.628008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606da2de-9414-43c9-9e31-a0e52be1fb55 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:12.632830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a65fe-1574-4f6c-8e28-6e18f31586bb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:12.668839 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23ee92-8a98-48ce-9ff1-466a4610a08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:12.669686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23ee92-8a98-48ce-9ff1-466a4610a08b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.669971 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23ee92-8a98-48ce-9ff1-466a4610a08b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.670256 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23ee92-8a98-48ce-9ff1-466a4610a08b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.681844 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23ee92-8a98-48ce-9ff1-466a4610a08b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:12.683313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23ee92-8a98-48ce-9ff1-466a4610a08b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:12.690108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:12.691060 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.691548 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.692034 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.692956 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:12.693497 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:12.695083 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-9e23ee92-8a98-48ce-9ff1-466a4610a08b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:12.701813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:12.702802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:12.703773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:12.703773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:12.703773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:12.703773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:12.703773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 381/798] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:12.704618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f64b14-e472-481b-8a70-da33d50099d3 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.704904 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f64b14-e472-481b-8a70-da33d50099d3 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.709714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f64b14-e472-481b-8a70-da33d50099d3 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.710947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 418/799] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:12.716173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:12.716318 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 382/800] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:12.717395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09907192-c335-4c84-b7b0-d1de10bcbe5f None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.717680 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09907192-c335-4c84-b7b0-d1de10bcbe5f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.717961 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09907192-c335-4c84-b7b0-d1de10bcbe5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.736262 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-09907192-c335-4c84-b7b0-d1de10bcbe5f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:12.737882 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-09907192-c335-4c84-b7b0-d1de10bcbe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:12.805617 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 383/801] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:12.807289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09907192-c335-4c84-b7b0-d1de10bcbe5f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:12.845730 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103388d-e67c-4f88-bb25-db758d59dd34 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:12.880459 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:12.881501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.882018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.882530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.883468 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:12.883996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:12.885610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:12.898968 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 419/802] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:12.900151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:12.938377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09907192-c335-4c84-b7b0-d1de10bcbe5f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:12.977099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:12.978305 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:12.978816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:12.979343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:12.987005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:12.988750 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:13.003533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:13.004526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:13.005530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:13.005530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:13.005530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:13.005530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:13.005530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:13.005530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:13.005530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:13.005530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:13.007809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9daa5-dd50-4544-9531-6fd52331e39a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.013197 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 384/803] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:13.013197 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.042309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2e616-4e5a-4d82-834e-4165ee8fc2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.043108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e616-4e5a-4d82-834e-4165ee8fc2a0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.043391 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e616-4e5a-4d82-834e-4165ee8fc2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.043658 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e616-4e5a-4d82-834e-4165ee8fc2a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.055836 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2e616-4e5a-4d82-834e-4165ee8fc2a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-e6c2e616-4e5a-4d82-834e-4165ee8fc2a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:13.063696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:13.065205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:13.066124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 420/804] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:13.066879 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c6f5-5c57-4d2a-ba8b-acd1921b302d None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.066879 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c6f5-5c57-4d2a-ba8b-acd1921b302d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.066879 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7c6f5-5c57-4d2a-ba8b-acd1921b302d None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.068187 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 421/805] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:13.069084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7c6f5-5c57-4d2a-ba8b-acd1921b302d None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.131938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82357e7-56d6-45e6-84e5-84472de5d185 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.165766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02cc6f-6754-4e2d-bed0-f5af5ee5add8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.166564 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02cc6f-6754-4e2d-bed0-f5af5ee5add8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.166848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02cc6f-6754-4e2d-bed0-f5af5ee5add8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.167125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02cc6f-6754-4e2d-bed0-f5af5ee5add8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.173760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02cc6f-6754-4e2d-bed0-f5af5ee5add8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.175215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02cc6f-6754-4e2d-bed0-f5af5ee5add8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.183905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7c6f5-5c57-4d2a-ba8b-acd1921b302d None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-2d02cc6f-6754-4e2d-bed0-f5af5ee5add8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:13.189783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:13.190755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:13.191731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 385/806] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:13.192288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11135a4c-d020-49f4-aea1-3c2a71d32a3f None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.192843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11135a4c-d020-49f4-aea1-3c2a71d32a3f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.192843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11135a4c-d020-49f4-aea1-3c2a71d32a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.210096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-11135a4c-d020-49f4-aea1-3c2a71d32a3f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:13.211290 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-11135a4c-d020-49f4-aea1-3c2a71d32a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:13.221904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.222568 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.222851 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.223130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.223914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:13.224228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.225717 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.238205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 422/807] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:13.238899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.288956 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 386/808] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:13.290137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11135a4c-d020-49f4-aea1-3c2a71d32a3f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.344295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f20445-a132-4408-b7f8-5b05e1bca74a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.377053 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.377725 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.378009 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.378282 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.379026 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:13.379334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.380698 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.383703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11135a4c-d020-49f4-aea1-3c2a71d32a3f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.388434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 423/809] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:13.389229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.421293 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d493793-e7ae-4018-8242-5a26cd453332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.421967 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d493793-e7ae-4018-8242-5a26cd453332 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.422250 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d493793-e7ae-4018-8242-5a26cd453332 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d493793-e7ae-4018-8242-5a26cd453332 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.423335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d493793-e7ae-4018-8242-5a26cd453332 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:13.423645 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d493793-e7ae-4018-8242-5a26cd453332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.425100 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d493793-e7ae-4018-8242-5a26cd453332 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.437298 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 387/810] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:13.438638 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc201ebb-9069-4780-ae88-c672f830b1c3 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.438936 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc201ebb-9069-4780-ae88-c672f830b1c3 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.439209 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc201ebb-9069-4780-ae88-c672f830b1c3 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.440467 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 388/811] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:13.441193 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc201ebb-9069-4780-ae88-c672f830b1c3 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.494297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6f71-181d-4101-8080-00d425712f35 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.525665 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da9274-318e-42c8-9a29-c7bd21274286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.526543 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da9274-318e-42c8-9a29-c7bd21274286 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.526825 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da9274-318e-42c8-9a29-c7bd21274286 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.527097 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da9274-318e-42c8-9a29-c7bd21274286 None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/e28363f2360842958bfe92394afe9651/roles/05908f87badf453aaf2ee23a431445f0` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.540872 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc201ebb-9069-4780-ae88-c672f830b1c3 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.541312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da9274-318e-42c8-9a29-c7bd21274286 None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=e28363f2360842958bfe92394afe9651, role_id=05908f87badf453aaf2ee23a431445f0)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.542749 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da9274-318e-42c8-9a29-c7bd21274286 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.554882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 424/812] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:16:13 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/e28363f2360842958bfe92394afe9651/roles/05908f87badf453aaf2ee23a431445f0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:16:13.555671 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da9274-318e-42c8-9a29-c7bd21274286 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.593438 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.594581 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.595027 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.595460 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] PATH_INFO: `/v3/services/network` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.596299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.598590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:13.616121 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:13.617630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:16:13.619031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:13.621047 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 389/813] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:13.622055 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.669432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da9274-318e-42c8-9a29-c7bd21274286 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.705237 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66778c14-a84d-42f1-9e1a-8cad13ba29d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.706052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66778c14-a84d-42f1-9e1a-8cad13ba29d6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.706338 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66778c14-a84d-42f1-9e1a-8cad13ba29d6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.706612 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66778c14-a84d-42f1-9e1a-8cad13ba29d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.717873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66778c14-a84d-42f1-9e1a-8cad13ba29d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.719357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66778c14-a84d-42f1-9e1a-8cad13ba29d6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.727011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50bcc0-6b2f-4fe0-b945-4b7d321e3c5a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-66778c14-a84d-42f1-9e1a-8cad13ba29d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:13.739806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:13.740865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:13.742004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:13.742004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:13.742004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:13.742004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:13.742004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 425/814] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:13.746666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c34ec3-b300-4d80-a6f4-84b39e22231e None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.746666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c34ec3-b300-4d80-a6f4-84b39e22231e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.746666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c34ec3-b300-4d80-a6f4-84b39e22231e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.762959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970fd61-a855-479b-a510-4284c25c55db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.763334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-b0c34ec3-b300-4d80-a6f4-84b39e22231e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:13.763839 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970fd61-a855-479b-a510-4284c25c55db None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.764128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970fd61-a855-479b-a510-4284c25c55db None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.764413 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970fd61-a855-479b-a510-4284c25c55db None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/e81aea4ac1ae45e2997c6c70e8c0028a/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.764519 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-b0c34ec3-b300-4d80-a6f4-84b39e22231e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:13.766464 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970fd61-a855-479b-a510-4284c25c55db None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=e81aea4ac1ae45e2997c6c70e8c0028a, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.767903 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970fd61-a855-479b-a510-4284c25c55db None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.781513 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 390/815] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:16:13 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/e81aea4ac1ae45e2997c6c70e8c0028a/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:16:13.782464 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970fd61-a855-479b-a510-4284c25c55db None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.865068 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 426/816] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:13.865068 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c34ec3-b300-4d80-a6f4-84b39e22231e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:13.929838 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970fd61-a855-479b-a510-4284c25c55db None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:13.968217 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:13.969014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:13.969448 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:13.969801 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:13.970631 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] RBAC: Adding query filter params (name=network) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:13.971017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:13.972510 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:13.979220 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 391/817] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:13.980181 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.037695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c34ec3-b300-4d80-a6f4-84b39e22231e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.104267 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712f27-a0d4-4712-b34d-9d66aae314e4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.141347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.142033 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.142317 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.142594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.142965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.143327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:14.143632 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:14.144441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.144987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.145134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:14.145626 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.146693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:14.149506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:14.164021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 427/818] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:14.164021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:14.173531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:14.174570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:16:14.175489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:14.180563 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 392/819] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:16:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:14.181431 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.291018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510781-8a02-485f-8f79-fe1697d36fac None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.349888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.350572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.351680 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.351984 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.352714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] RBAC: Adding query filter params (type=network) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:14.353024 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:14.354435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:14.368226 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 428/820] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:14.371965 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.373566 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4989a-8789-4cfb-9101-d77e3c101d41 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.434226 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.435774 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.436254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.437093 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] PATH_INFO: `/v3/users/nova` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.447295 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:14.449771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:14.468932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:14.470024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:14.471056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:14.478917 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 393/821] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:16:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:14.478917 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.502800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403d5af-e8d2-4844-8a08-13fdfe480a05 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.544930 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.546168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.546815 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.547293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.548736 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:14.549289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:14.550841 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:14.557671 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 429/822] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:16:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:14.558748 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.588395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e2258-f9f2-4dfb-badf-613a72ed0bca None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.622453 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823aed1-fb5a-4078-bcb5-e65d9e17b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.623458 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823aed1-fb5a-4078-bcb5-e65d9e17b174 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.623930 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823aed1-fb5a-4078-bcb5-e65d9e17b174 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.624398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823aed1-fb5a-4078-bcb5-e65d9e17b174 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.625409 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823aed1-fb5a-4078-bcb5-e65d9e17b174 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:14.627039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823aed1-fb5a-4078-bcb5-e65d9e17b174 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:14.653913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 394/823] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:16:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:14.655556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823aed1-fb5a-4078-bcb5-e65d9e17b174 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.675361 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6d5a-8146-45d2-9363-0578a9832b79 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.731378 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.733385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.735519 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.736032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.737228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:14.737959 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:14.740327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:14.767487 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 430/824] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:16:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:14.773153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823aed1-fb5a-4078-bcb5-e65d9e17b174 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.776693 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.812681 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.813717 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.813995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.814768 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:14.815134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:14.816530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:14.822092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 395/825] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:16:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:14.831905 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:14.894826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8eabd-d486-4855-8844-26d98cb79785 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:14.937488 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a451e43-590b-4f50-aa71-44c06e934fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:14.938620 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a451e43-590b-4f50-aa71-44c06e934fb7 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:14.939140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a451e43-590b-4f50-aa71-44c06e934fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:14.940257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a451e43-590b-4f50-aa71-44c06e934fb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:14.988467 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694a9b7-83ca-46d5-afd2-7531dfa0b3b5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:15.030165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a451e43-590b-4f50-aa71-44c06e934fb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:15.030534 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b8b6b-e00a-414b-8648-a9e8eef44800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:15.031205 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b8b6b-e00a-414b-8648-a9e8eef44800 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:15.031491 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b8b6b-e00a-414b-8648-a9e8eef44800 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:15.031744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a451e43-590b-4f50-aa71-44c06e934fb7 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:15.031860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b8b6b-e00a-414b-8648-a9e8eef44800 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:15.032498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b8b6b-e00a-414b-8648-a9e8eef44800 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:15.034071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b8b6b-e00a-414b-8648-a9e8eef44800 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:15.060329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 396/826] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:16:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:15.166830 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 431/827] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:16:15.172295 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b8b6b-e00a-414b-8648-a9e8eef44800 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:15.308326 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b8b6b-e00a-414b-8648-a9e8eef44800 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:15.343969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:15.344775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:15.345083 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:15.345367 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:15.356800 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:15.358367 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:15.376682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:15.377744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:15.378715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:15.378715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:15.378715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:15.378715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:15.378715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 397/828] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:15.384996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a451e43-590b-4f50-aa71-44c06e934fb7 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:15.487566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a451e43-590b-4f50-aa71-44c06e934fb7 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:15.522397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:15.523015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:15.523300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:15.523581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:15.524312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:15.524621 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:15.526035 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:15.537923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 432/829] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:15.543922 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:15.644689 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3cbc9-0a95-4338-9b2b-7e6f176d2144 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:15.680450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:15.681703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:15.682230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:15.682731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:15.690106 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:15.691847 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:15.706362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:15.707352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:15.708359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:15.708359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:15.708359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:15.708359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:15.708359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:15.708359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:15.708359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:15.708359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:15.711486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 398/830] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:15.718159 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:15.822011 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dec79-4c1e-458f-a5a2-4f04419e71f6 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:15.860298 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc2bb4-c9d7-4509-af3f-a01090203b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:15.860971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc2bb4-c9d7-4509-af3f-a01090203b01 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:15.861292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc2bb4-c9d7-4509-af3f-a01090203b01 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:15.861575 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc2bb4-c9d7-4509-af3f-a01090203b01 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:15.862370 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc2bb4-c9d7-4509-af3f-a01090203b01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:15.862686 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc2bb4-c9d7-4509-af3f-a01090203b01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:15.864222 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc2bb4-c9d7-4509-af3f-a01090203b01 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:15.877213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 433/831] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:15.883210 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:15.990618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e7457-2f09-40a2-8efd-410c0436fd2b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:16.015247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfefbe-efe4-465f-bdc8-b670d31c1553 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.015566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfefbe-efe4-465f-bdc8-b670d31c1553 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.015820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfefbe-efe4-465f-bdc8-b670d31c1553 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.017510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 434/832] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:16.022473 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f3ee1-0ba7-42d2-b82b-bd7dd5a2f0b9 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.023294 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f3ee1-0ba7-42d2-b82b-bd7dd5a2f0b9 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.023576 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f3ee1-0ba7-42d2-b82b-bd7dd5a2f0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.028152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:16.029100 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.029612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.030095 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.031144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=f3a73bc43c23400d8ee7d4e880aa7242) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:16.031679 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:16.033367 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:16.040677 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 399/833] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:15 2022] GET /identity/v3/role_assignments?user.id=f3a73bc43c23400d8ee7d4e880aa7242&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:16.045032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-7a1f3ee1-0ba7-42d2-b82b-bd7dd5a2f0b9 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:16.045032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-7a1f3ee1-0ba7-42d2-b82b-bd7dd5a2f0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:16.119942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 435/834] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:16.143747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:16.254693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c741d-255b-42d9-8fe9-b1c853a46e31 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:16.291471 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef109cd-e571-4994-9021-77bd488eb8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:16.292277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef109cd-e571-4994-9021-77bd488eb8e0 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.292563 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef109cd-e571-4994-9021-77bd488eb8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.292839 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef109cd-e571-4994-9021-77bd488eb8e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.303252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef109cd-e571-4994-9021-77bd488eb8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-6ef109cd-e571-4994-9021-77bd488eb8e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:16.311660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:16.312711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:16.313683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 400/835] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:16:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:16.320772 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f3ee1-0ba7-42d2-b82b-bd7dd5a2f0b9 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:16.347425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39686e-be1b-4337-86a4-80ce53e03c44 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.347762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39686e-be1b-4337-86a4-80ce53e03c44 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.347987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39686e-be1b-4337-86a4-80ce53e03c44 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.349602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 401/836] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:16.355099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b5511-0a04-46c9-ab03-5730481979a1 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.355451 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b5511-0a04-46c9-ab03-5730481979a1 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.355663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b5511-0a04-46c9-ab03-5730481979a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.373577 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-f92b5511-0a04-46c9-ab03-5730481979a1 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:16.374690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-f92b5511-0a04-46c9-ab03-5730481979a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:16.434658 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f3ee1-0ba7-42d2-b82b-bd7dd5a2f0b9 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:16.443188 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 402/837] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:16.473132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b5511-0a04-46c9-ab03-5730481979a1 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:16.475741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:16.477392 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.477978 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.478467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.479433 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:16.480086 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:16.481845 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:16.491076 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 436/838] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:16:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:16.497104 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:16.572564 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b5511-0a04-46c9-ab03-5730481979a1 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:16.598669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5a28e-cfc8-4eb1-b29a-e43f7eec71c5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:16.607053 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a11563-3a3d-4d9b-8c35-776c6a7863be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:16.607853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a11563-3a3d-4d9b-8c35-776c6a7863be None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.608135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a11563-3a3d-4d9b-8c35-776c6a7863be None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.608411 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a11563-3a3d-4d9b-8c35-776c6a7863be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.618993 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a11563-3a3d-4d9b-8c35-776c6a7863be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f9a11563-3a3d-4d9b-8c35-776c6a7863be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:16.626848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:16.627859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:16.628798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 403/839] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:16.634429 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a11563-3a3d-4d9b-8c35-776c6a7863be None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:16.639087 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:16.640154 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.640651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.641229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.654586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:16.656353 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:16.674457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:16.675450 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:16.676455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:16.676455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:16.676455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:16.676455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:16.683158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 437/840] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:16.684383 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:16.764359 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a11563-3a3d-4d9b-8c35-776c6a7863be None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:16.796721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba0859-690b-4e86-8902-9669f52d785b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:16.808941 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:16.809659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.809945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.810221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.811155 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:16.811467 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:16.813552 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:16.825207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 404/841] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:16.830492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:16.840271 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:16.840943 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:16.841299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:16.841579 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:16.842319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:16.842625 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:16.844107 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:16.861366 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 438/842] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:16.862308 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:16.957614 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c527d7-4974-480a-949b-23b292d987ad None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:16.966866 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de006771-e0c8-4dac-8d2b-82b769170e01 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.002470 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.003314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.003597 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.003872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.005815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.008204 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.008738 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.009259 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.011663 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.013136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.023490 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.025276 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:17.030149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:17.031151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:17.032189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:17.036365 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 439/843] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:16:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:17.037252 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:17.049938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:17.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:17.051962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:17.051962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:17.051962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:17.051962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:17.058656 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 405/844] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:17.059631 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.171105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22e365-7c14-44c0-9db3-1f46d619cbb6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.177320 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37c0ef-80a9-4dce-b841-8d69674ba1b4 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.205486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.206191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.206498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.206730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.207550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:17.207919 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.209358 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.227122 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 406/845] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:17.228350 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.229420 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.230470 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.230992 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.231718 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.233886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:17.234447 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.236140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.258172 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 440/846] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:17.265610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.370492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d138b-d13d-4549-abe5-5ce6734361af None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.384740 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f741127-4b80-4805-aa31-863ea6fd15be None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.405707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fb14b-bc39-4d3f-a400-4d8f78bff2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.406289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fb14b-bc39-4d3f-a400-4d8f78bff2f4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.406570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fb14b-bc39-4d3f-a400-4d8f78bff2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.406841 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fb14b-bc39-4d3f-a400-4d8f78bff2f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.414409 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fb14b-bc39-4d3f-a400-4d8f78bff2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.415954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fb14b-bc39-4d3f-a400-4d8f78bff2f4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.420921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c4e5a-bcd6-4009-b9fb-981819c2fde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.421671 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c4e5a-bcd6-4009-b9fb-981819c2fde2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.421962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c4e5a-bcd6-4009-b9fb-981819c2fde2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.422238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c4e5a-bcd6-4009-b9fb-981819c2fde2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-223fb14b-bc39-4d3f-a400-4d8f78bff2f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:17.431547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:17.432668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:16:17.433751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:17.437375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 407/847] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:17.438976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9c519-c7be-4aef-bdff-c6beeca287ee None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.439279 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9c519-c7be-4aef-bdff-c6beeca287ee None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.439557 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9c519-c7be-4aef-bdff-c6beeca287ee None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.441072 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 408/848] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:17.446138 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9c519-c7be-4aef-bdff-c6beeca287ee None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.493742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c4e5a-bcd6-4009-b9fb-981819c2fde2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.495117 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c4e5a-bcd6-4009-b9fb-981819c2fde2 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.550832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9c519-c7be-4aef-bdff-c6beeca287ee None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.586418 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.587087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.587382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.587663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.588499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:17.588814 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.590263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.598472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 441/849] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:16:17.598472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b3d2d-113a-4b4d-a835-6c3583f5dee8 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.598472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b3d2d-113a-4b4d-a835-6c3583f5dee8 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.598472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b3d2d-113a-4b4d-a835-6c3583f5dee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.615086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 409/850] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:17.615086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.618658 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-976b3d2d-113a-4b4d-a835-6c3583f5dee8 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:17.619848 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-976b3d2d-113a-4b4d-a835-6c3583f5dee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:17.685298 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 442/851] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:17.686165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b3d2d-113a-4b4d-a835-6c3583f5dee8 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.720463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb37c91-9170-4705-aa48-b5e0938a53fa None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.754171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.754894 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.755177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.755454 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.765585 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.767044 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.782513 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b3d2d-113a-4b4d-a835-6c3583f5dee8 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:17.785802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:17.786758 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:17.792990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:17.792990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:17.792990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:17.792990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:17.792990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 410/852] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:17.792990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.820570 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32213643-5c10-4fbf-b70f-ef5514f53131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.821258 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213643-5c10-4fbf-b70f-ef5514f53131 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.821541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213643-5c10-4fbf-b70f-ef5514f53131 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.821819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32213643-5c10-4fbf-b70f-ef5514f53131 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.888915 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c0c81-b9fa-4c2a-9029-08bb1a50ae98 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:17.889969 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32213643-5c10-4fbf-b70f-ef5514f53131 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.891694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32213643-5c10-4fbf-b70f-ef5514f53131 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.923978 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:17.924785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:17.925096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:17.925384 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:17.926044 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:17.927402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:17.942441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:17.943627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:17.944562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 411/853] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:17.945493 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:17.998827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 443/854] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:16:18.000049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32213643-5c10-4fbf-b70f-ef5514f53131 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.044917 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d265b9-7ad6-45d7-9dae-1493692dd549 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.077628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.078297 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.078585 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.079138 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.079591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:18.079903 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.081314 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.094717 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 412/855] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:18.095928 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.102453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32213643-5c10-4fbf-b70f-ef5514f53131 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.135327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.136121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.136508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.136893 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.137731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:18.138157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.139620 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.144205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 444/856] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:18.150033 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.195291 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcd922-29ad-4d83-a89e-73fa899b348b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.232320 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.233258 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.233544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.233822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.244216 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.245715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.259018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349e8da-d7c4-4a1a-bcaa-4d88871ef7b9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:18.266133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:18.267378 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:18.268583 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:18.268583 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:18.268583 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:18.268583 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:18.269189 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 413/857] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:18.273137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.302194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.302924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.303197 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.303475 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.310096 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.311539 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:18.325770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:18.326756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 445/858] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:18.327729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.378020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d35d40-fc21-4a1d-818b-1ca2bbe1f495 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.411456 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.412115 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.412397 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.412671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.413438 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:18.413739 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.415096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.419305 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 414/859] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:18.420061 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.438850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadfb7-61d2-48af-9f1e-e9affe954e93 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.476155 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c2227-e45a-41c8-97f4-6c9b22bafac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.476838 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c2227-e45a-41c8-97f4-6c9b22bafac9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.477149 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c2227-e45a-41c8-97f4-6c9b22bafac9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.477432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c2227-e45a-41c8-97f4-6c9b22bafac9 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.478177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c2227-e45a-41c8-97f4-6c9b22bafac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:18.478524 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c2227-e45a-41c8-97f4-6c9b22bafac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.479892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c2227-e45a-41c8-97f4-6c9b22bafac9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.492826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 446/860] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:18.494589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a3b2a-6692-4dd5-869a-7f63aa362bd3 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.494878 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a3b2a-6692-4dd5-869a-7f63aa362bd3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.495157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a3b2a-6692-4dd5-869a-7f63aa362bd3 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.501001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 447/861] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:18.502020 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a3b2a-6692-4dd5-869a-7f63aa362bd3 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.545665 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f4f0e-463d-4e28-b599-1d70e679e4f6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.582940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2974e3d-8a3e-4c44-8360-67c5146f3271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.583655 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974e3d-8a3e-4c44-8360-67c5146f3271 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.584006 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974e3d-8a3e-4c44-8360-67c5146f3271 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.584665 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974e3d-8a3e-4c44-8360-67c5146f3271 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.586171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2974e3d-8a3e-4c44-8360-67c5146f3271 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:18.586551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2974e3d-8a3e-4c44-8360-67c5146f3271 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.588076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2974e3d-8a3e-4c44-8360-67c5146f3271 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.599923 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 415/862] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:18.601258 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325025a8-52b5-45e2-bab8-adaa0076b556 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.601549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325025a8-52b5-45e2-bab8-adaa0076b556 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.601829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325025a8-52b5-45e2-bab8-adaa0076b556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.613247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a3b2a-6692-4dd5-869a-7f63aa362bd3 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.619573 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-325025a8-52b5-45e2-bab8-adaa0076b556 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:18.620762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-325025a8-52b5-45e2-bab8-adaa0076b556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:18.649943 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.650733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.651063 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.651334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.658035 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.659466 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:18.673241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:18.674405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:18.675572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:18.675572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:18.675572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:18.675572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:18.675572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:18.675572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:18.675572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:18.675572 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:18.678582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 448/863] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:18.678582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.697124 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 416/864] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:18.697124 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325025a8-52b5-45e2-bab8-adaa0076b556 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.786555 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325025a8-52b5-45e2-bab8-adaa0076b556 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.788917 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287e1bc-af08-4010-a3b7-5eaf16ac784e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.820487 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.821191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.821479 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.821760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.822092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.822550 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:18.822747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.822876 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.823046 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.823296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.824125 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=e28363f2360842958bfe92394afe9651) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:18.824305 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.824471 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.825879 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.831839 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 449/865] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/role_assignments?user.id=e28363f2360842958bfe92394afe9651&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:18.832465 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.851766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 417/866] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:18.853155 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:18.945364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08642787-1da1-4b9e-a43d-f7660e920654 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.962971 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658173c8-e630-4676-9c59-735837b8bc17 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:18.981947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:18.983044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:18.983330 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:18.983612 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:18.984280 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:18.985721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:18.999485 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:19.000616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:19.001656 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 450/867] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:19.007629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:19.010029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:19.010710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:19.011035 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:19.011322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:19.012165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=e81aea4ac1ae45e2997c6c70e8c0028a) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:19.012485 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:19.014020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:19.020879 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 418/868] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:18 2022] GET /identity/v3/role_assignments?user.id=e81aea4ac1ae45e2997c6c70e8c0028a&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:19.124160 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1f9d1-6b17-47ff-b583-854849bdc166 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:19.161732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0bbea-b0a1-41a0-9e29-647f44140bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:19.162482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0bbea-b0a1-41a0-9e29-647f44140bb9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:19.162744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0bbea-b0a1-41a0-9e29-647f44140bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:19.163017 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0bbea-b0a1-41a0-9e29-647f44140bb9 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:19.164342 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0bbea-b0a1-41a0-9e29-647f44140bb9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:19.164661 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0bbea-b0a1-41a0-9e29-647f44140bb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:19.166064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0bbea-b0a1-41a0-9e29-647f44140bb9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:19.171677 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 451/869] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:19.177330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:19.286852 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08af18-f798-43dc-8fd7-e88676408677 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:19.321075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efce27-a794-41b0-bd3b-8e73d663c95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:19.321745 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efce27-a794-41b0-bd3b-8e73d663c95a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:19.322029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efce27-a794-41b0-bd3b-8e73d663c95a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:19.322304 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efce27-a794-41b0-bd3b-8e73d663c95a None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:19.323013 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efce27-a794-41b0-bd3b-8e73d663c95a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:19.323320 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efce27-a794-41b0-bd3b-8e73d663c95a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:19.324683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efce27-a794-41b0-bd3b-8e73d663c95a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:19.329486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 419/870] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:20.680927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69cde-35d9-4997-8291-7bd0f0d7d391 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:20.681534 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69cde-35d9-4997-8291-7bd0f0d7d391 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:20.681534 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d69cde-35d9-4997-8291-7bd0f0d7d391 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:20.683122 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 452/871] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:20.688358 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c3185-73b8-451b-ac4a-a9c794f70c20 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:20.688762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c3185-73b8-451b-ac4a-a9c794f70c20 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:20.688960 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c3185-73b8-451b-ac4a-a9c794f70c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:20.706134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-671c3185-73b8-451b-ac4a-a9c794f70c20 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:20.707309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-671c3185-73b8-451b-ac4a-a9c794f70c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:20.775256 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 420/872] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:20.797236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d69cde-35d9-4997-8291-7bd0f0d7d391 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:20.902939 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d69cde-35d9-4997-8291-7bd0f0d7d391 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:20.937616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9308ca-7a91-4f9a-acc5-9ca75e89f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:20.938201 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9308ca-7a91-4f9a-acc5-9ca75e89f127 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:20.938479 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9308ca-7a91-4f9a-acc5-9ca75e89f127 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:20.938751 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9308ca-7a91-4f9a-acc5-9ca75e89f127 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:20.939488 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9308ca-7a91-4f9a-acc5-9ca75e89f127 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:20.940846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9308ca-7a91-4f9a-acc5-9ca75e89f127 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:20.956904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 453/873] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:16:20 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:21.292120 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea867-3eb1-4c01-9d8e-d24ae3821499 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.292479 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea867-3eb1-4c01-9d8e-d24ae3821499 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.292681 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea867-3eb1-4c01-9d8e-d24ae3821499 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.294363 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 421/874] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:21.299759 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9cff8-4706-44e9-a1a6-b605286672a4 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.300052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9cff8-4706-44e9-a1a6-b605286672a4 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.300331 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9cff8-4706-44e9-a1a6-b605286672a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.317353 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-65b9cff8-4706-44e9-a1a6-b605286672a4 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:21.318533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-65b9cff8-4706-44e9-a1a6-b605286672a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:21.354111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec176a7-2b2d-4183-92b6-8257cebc43a5 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.354747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec176a7-2b2d-4183-92b6-8257cebc43a5 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.355234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec176a7-2b2d-4183-92b6-8257cebc43a5 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.358328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 422/875] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:21.362707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ca637-5e18-4791-911b-cba0aec28017 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.363237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ca637-5e18-4791-911b-cba0aec28017 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.363748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ca637-5e18-4791-911b-cba0aec28017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.380595 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-cf5ca637-5e18-4791-911b-cba0aec28017 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:21.382725 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-cf5ca637-5e18-4791-911b-cba0aec28017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:21.399829 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 454/876] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:21.422589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9cff8-4706-44e9-a1a6-b605286672a4 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:21.475037 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 423/877] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:21.493686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ca637-5e18-4791-911b-cba0aec28017 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:21.535195 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9cff8-4706-44e9-a1a6-b605286672a4 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:21.570512 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a79fd-1121-4eb5-aa9b-bddfb2cfb42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:21.571331 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a79fd-1121-4eb5-aa9b-bddfb2cfb42b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.571634 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a79fd-1121-4eb5-aa9b-bddfb2cfb42b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.572138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a79fd-1121-4eb5-aa9b-bddfb2cfb42b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.582402 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a79fd-1121-4eb5-aa9b-bddfb2cfb42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-657a79fd-1121-4eb5-aa9b-bddfb2cfb42b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:21.590551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:21.591581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:21.592505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:21.596879 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 455/878] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:16:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:21.597762 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a79fd-1121-4eb5-aa9b-bddfb2cfb42b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:21.599584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ca637-5e18-4791-911b-cba0aec28017 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:21.634836 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cb065-6627-4c25-be2d-fc0e348a33c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:21.635637 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb065-6627-4c25-be2d-fc0e348a33c1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.635913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb065-6627-4c25-be2d-fc0e348a33c1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.636183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb065-6627-4c25-be2d-fc0e348a33c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.636832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cb065-6627-4c25-be2d-fc0e348a33c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:21.638218 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cb065-6627-4c25-be2d-fc0e348a33c1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-896cb065-6627-4c25-be2d-fc0e348a33c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:21.650153 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:21.651443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:21.652476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 424/879] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:21.653426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092214e6-4ae4-4a23-a6f9-c545b76b725f None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.653426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092214e6-4ae4-4a23-a6f9-c545b76b725f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.653426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092214e6-4ae4-4a23-a6f9-c545b76b725f None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.677588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 425/880] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:21.677588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092214e6-4ae4-4a23-a6f9-c545b76b725f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:21.725573 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a79fd-1121-4eb5-aa9b-bddfb2cfb42b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:21.758547 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c33bd8-1314-4e63-a00e-ec561eda0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:21.759222 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c33bd8-1314-4e63-a00e-ec561eda0de9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.759502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c33bd8-1314-4e63-a00e-ec561eda0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.759775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c33bd8-1314-4e63-a00e-ec561eda0de9 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.760530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c33bd8-1314-4e63-a00e-ec561eda0de9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:21.760833 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c33bd8-1314-4e63-a00e-ec561eda0de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:21.762228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c33bd8-1314-4e63-a00e-ec561eda0de9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:21.770044 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092214e6-4ae4-4a23-a6f9-c545b76b725f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:21.771356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 456/881] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:16:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:21.773363 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a64b13-7651-4504-8a58-c9b4b2756e48 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.773878 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a64b13-7651-4504-8a58-c9b4b2756e48 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.774347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a64b13-7651-4504-8a58-c9b4b2756e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.792749 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-b7a64b13-7651-4504-8a58-c9b4b2756e48 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:21.793866 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-b7a64b13-7651-4504-8a58-c9b4b2756e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:21.806945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:21.807988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.808281 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.808825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.809586 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:21.810054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:21.811731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:21.816761 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 426/882] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:21.817778 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:21.873436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 457/883] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:21.874359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a64b13-7651-4504-8a58-c9b4b2756e48 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:21.914819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d478c5a-c1d7-42bc-bca8-11377561d6a4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:21.947322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:21.948112 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:21.948391 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:21.948663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:21.958857 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:21.960820 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:21.972859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a64b13-7651-4504-8a58-c9b4b2756e48 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:21.978023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:21.978752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:21.979545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:21.979545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:21.979545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:21.979545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:21.979545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 427/884] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:21.980296 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:22.010198 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:22.010859 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:22.011139 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:22.011413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:22.012118 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:22.012422 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:22.013812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:22.018113 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 458/885] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:22.019213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:22.084084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdbb6d-713f-434e-9881-08a0e45c1bca None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:22.126309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f7e53-73ce-4c78-b754-08e9f2e2235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:22.126992 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f7e53-73ce-4c78-b754-08e9f2e2235e None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:22.127278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f7e53-73ce-4c78-b754-08e9f2e2235e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:22.127552 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f7e53-73ce-4c78-b754-08e9f2e2235e None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:22.128286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f7e53-73ce-4c78-b754-08e9f2e2235e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:22.129747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f7e53-73ce-4c78-b754-08e9f2e2235e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:22.140038 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08523b-a6b1-49dd-bbb7-3c9e3acc77ed None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:22.150159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 428/886] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:16:21 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:22.199757 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:22.200411 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:22.200699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:22.200973 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:22.201751 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:22.202058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:22.203533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:22.222291 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 459/887] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:22.224013 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f7e53-73ce-4c78-b754-08e9f2e2235e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:22.339421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f7e53-73ce-4c78-b754-08e9f2e2235e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:22.374734 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:22.375635 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:22.375919 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:22.376193 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:22.384608 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:22.386085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:22.400868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:22.401747 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:22.402506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:22.405125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 429/888] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:16:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:22.408052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:22.518691 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c3bac-4324-48ce-ad21-b27b069ad8fb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:22.553370 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:22.554392 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:22.554874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:22.555346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:22.556400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:22.556906 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:22.558566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:22.577012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 460/889] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:16:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:22.582645 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:22.686064 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefa28c-0fda-4814-ba18-247aea1ccdae None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:22.720630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d640b-8cfe-4e0f-9825-aa1a438ce9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:22.721645 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d640b-8cfe-4e0f-9825-aa1a438ce9a7 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:22.722134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d640b-8cfe-4e0f-9825-aa1a438ce9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:22.722631 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d640b-8cfe-4e0f-9825-aa1a438ce9a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:22.792558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d640b-8cfe-4e0f-9825-aa1a438ce9a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:22.795810 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d640b-8cfe-4e0f-9825-aa1a438ce9a7 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:22.894465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 430/890] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:22 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:16:22.900525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:22.999780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27714c-46d8-42f5-9ea6-416de60b0031 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:23.033513 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:23.034478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.034766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.035042 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.045400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:23.046777 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:23.064064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:23.064899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:23.065649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:23.065649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:23.065649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:23.065649 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:23.069173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 461/891] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:23.071833 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d640b-8cfe-4e0f-9825-aa1a438ce9a7 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:23.181206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d640b-8cfe-4e0f-9825-aa1a438ce9a7 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:23.218538 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515037de-fe66-49dd-81f8-52795838b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:23.219598 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515037de-fe66-49dd-81f8-52795838b256 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.220074 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515037de-fe66-49dd-81f8-52795838b256 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.220537 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515037de-fe66-49dd-81f8-52795838b256 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.221488 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515037de-fe66-49dd-81f8-52795838b256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:23.221993 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515037de-fe66-49dd-81f8-52795838b256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:23.223562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515037de-fe66-49dd-81f8-52795838b256 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:23.236817 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 431/892] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:23.249062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:23.342269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ff33c-5d4c-4862-b01c-90c2b153a36b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:23.355755 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb25b3a-2559-4c11-886e-45b244310621 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.356042 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb25b3a-2559-4c11-886e-45b244310621 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.356321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb25b3a-2559-4c11-886e-45b244310621 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.361782 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 432/893] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:23.363508 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388f8f2-8158-4f21-b717-124001fd75aa None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.363798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388f8f2-8158-4f21-b717-124001fd75aa None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.364082 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388f8f2-8158-4f21-b717-124001fd75aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.386312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:23.386499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-9388f8f2-8158-4f21-b717-124001fd75aa None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:23.387565 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.387686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-9388f8f2-8158-4f21-b717-124001fd75aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:23.388150 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.388614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.397309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:23.399707 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:23.417467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:23.418207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:23.418931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:23.418931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:23.418931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:23.418931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:23.418931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:23.418931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:23.418931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:23.418931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:23.421606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 462/894] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:23.427567 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:23.485062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 433/895] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:23.503141 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388f8f2-8158-4f21-b717-124001fd75aa None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:23.542052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51971f-5ca3-4cd4-9d1e-7f15c3ba4b23 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:23.576043 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:23.577014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.577514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.578003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.578975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:23.579477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:23.581211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:23.599529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 463/896] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:23.601170 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:23.628953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388f8f2-8158-4f21-b717-124001fd75aa None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:23.671849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:23.672648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.672925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.673240 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.673892 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:23.675388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:23.687239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:23.688078 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:23.694976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 434/897] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:23.697462 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:23.708788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a8b10-f639-4c22-bad9-33da440da2fc None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:23.748400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:23.749442 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.749926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.750397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.751441 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] RBAC: Adding query filter params (role.id=5bdec21d519c4a929fc1ddde15a7b9ca, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=e28363f2360842958bfe92394afe9651) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:23.751951 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:23.753619 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:23.769218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 464/898] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:23 2022] GET /identity/v3/role_assignments?user.id=e28363f2360842958bfe92394afe9651&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=5bdec21d519c4a929fc1ddde15a7b9ca => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:23.820693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab3460-25e3-4c7e-a467-453bef7fd752 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:23.860000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:23.860597 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:23.860876 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:23.861263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:23.861986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:23.862291 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:23.863682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:23.869743 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 435/899] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:23.875986 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:23.987878 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da53902-0e71-421c-9bdc-d319cedb327b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:24.023604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:24.024269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.024542 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.024809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.025536 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:24.025838 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:24.027192 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:24.032559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 465/900] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:24.038802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:24.144211 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623e93-25dd-4c87-9c7f-1a61aa388d2b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:24.183000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883f83c-4861-46f6-9202-bb23b1576974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:24.183675 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f83c-4861-46f6-9202-bb23b1576974 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.183969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f83c-4861-46f6-9202-bb23b1576974 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.185624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f83c-4861-46f6-9202-bb23b1576974 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.186438 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883f83c-4861-46f6-9202-bb23b1576974 None admin] RBAC: Adding query filter params (interface=public, service_id=880eeb5aa2f0419a9c6470b5ee099230, region_id=RegionOne) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:24.186752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883f83c-4861-46f6-9202-bb23b1576974 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:24.188126 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883f83c-4861-46f6-9202-bb23b1576974 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:24.194191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 436/901] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:16:24 2022] GET /identity/v3/endpoints?service_id=880eeb5aa2f0419a9c6470b5ee099230&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:24.202341 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:24.305290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6a01-189f-4577-9594-4ca90842528b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:24.347658 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf766a04-e335-43d6-9e8e-88ddf7763939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:24.348336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766a04-e335-43d6-9e8e-88ddf7763939 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.348618 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766a04-e335-43d6-9e8e-88ddf7763939 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.348893 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766a04-e335-43d6-9e8e-88ddf7763939 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.349612 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf766a04-e335-43d6-9e8e-88ddf7763939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:24.351115 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf766a04-e335-43d6-9e8e-88ddf7763939 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:24.358247 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 466/902] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:16:24 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:16:24.362843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d749-c7ab-47f0-a5a4-492459bd8963 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.363461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d749-c7ab-47f0-a5a4-492459bd8963 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.363927 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d749-c7ab-47f0-a5a4-492459bd8963 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.366009 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 437/903] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:24.372594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece282a-fa25-4ca2-965f-bab11f6ad2cc None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.373293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece282a-fa25-4ca2-965f-bab11f6ad2cc None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.373874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece282a-fa25-4ca2-965f-bab11f6ad2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.392493 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-0ece282a-fa25-4ca2-965f-bab11f6ad2cc None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:24.394090 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-0ece282a-fa25-4ca2-965f-bab11f6ad2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:24.468208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 467/904] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:24.490806 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9d749-c7ab-47f0-a5a4-492459bd8963 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:24.511823 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a2456-cd13-4002-a2d2-c3e30b779117 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.512124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a2456-cd13-4002-a2d2-c3e30b779117 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.512392 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a2456-cd13-4002-a2d2-c3e30b779117 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.514172 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 468/905] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:24.519815 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798ed8d-3db0-4730-9ed9-c42baedfae8b None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.520090 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798ed8d-3db0-4730-9ed9-c42baedfae8b None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.520372 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798ed8d-3db0-4730-9ed9-c42baedfae8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.543433 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-a798ed8d-3db0-4730-9ed9-c42baedfae8b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:24.545347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-a798ed8d-3db0-4730-9ed9-c42baedfae8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:24.619362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9d749-c7ab-47f0-a5a4-492459bd8963 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:24.621731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 469/906] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:24.648727 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798ed8d-3db0-4730-9ed9-c42baedfae8b None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:24.655368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbfe2d-7bdc-4ecb-93b4-d80b2144870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:24.656042 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfe2d-7bdc-4ecb-93b4-d80b2144870b None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.656328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfe2d-7bdc-4ecb-93b4-d80b2144870b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.656604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbfe2d-7bdc-4ecb-93b4-d80b2144870b None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.657820 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbfe2d-7bdc-4ecb-93b4-d80b2144870b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:24.659206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbfe2d-7bdc-4ecb-93b4-d80b2144870b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:24.679551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 438/907] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:16:24 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:24.765564 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798ed8d-3db0-4730-9ed9-c42baedfae8b None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:24.811258 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:24.812101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.812378 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.812652 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.813341 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:24.814725 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:24.828278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:24.829167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:24.829905 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:24.833940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 470/908] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:16:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:24.836912 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbfe2d-7bdc-4ecb-93b4-d80b2144870b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:24.945996 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbfe2d-7bdc-4ecb-93b4-d80b2144870b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:24.982746 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:24.983745 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:24.984202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:24.984648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:24.985563 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:24.986050 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:24.987638 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:24.997134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 439/909] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:24.999457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:25.103335 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7acc7a-77b9-4187-850b-2c8eeacec840 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:25.138165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:25.139305 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:25.139780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:25.140243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:25.141152 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:25.141632 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:25.143180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:25.153159 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 471/910] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:25.155746 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:25.260951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513259a-71a2-48b5-9b5d-f7479f1ec16a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:25.297160 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c64df8-482d-4f09-9b90-e3be92f4cf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:25.297828 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c64df8-482d-4f09-9b90-e3be92f4cf2a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:25.298114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c64df8-482d-4f09-9b90-e3be92f4cf2a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:25.298391 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c64df8-482d-4f09-9b90-e3be92f4cf2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:25.299182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c64df8-482d-4f09-9b90-e3be92f4cf2a None admin] RBAC: Adding query filter params (interface=public, service_id=eb3346caa1c24f3e913c2e645e7b14ef, region_id=RegionOne) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:25.299558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c64df8-482d-4f09-9b90-e3be92f4cf2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:25.300987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c64df8-482d-4f09-9b90-e3be92f4cf2a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:25.306508 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 440/911] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:16:25 2022] GET /identity/v3/endpoints?service_id=eb3346caa1c24f3e913c2e645e7b14ef&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:25.311966 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:25.415109 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7da071-5c19-47d3-94d2-b99dbcacf45b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:25.449892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca266a-3387-42b9-b773-2dd4af9b1a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:25.450918 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca266a-3387-42b9-b773-2dd4af9b1a04 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:25.451397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca266a-3387-42b9-b773-2dd4af9b1a04 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:25.451871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca266a-3387-42b9-b773-2dd4af9b1a04 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:25.452744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca266a-3387-42b9-b773-2dd4af9b1a04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:25.454344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca266a-3387-42b9-b773-2dd4af9b1a04 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:25.464273 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 472/912] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:16:25 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:16:26.105488 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea1d82-3529-4767-85ac-fb7c85c0e68f None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.105983 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea1d82-3529-4767-85ac-fb7c85c0e68f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.106052 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea1d82-3529-4767-85ac-fb7c85c0e68f None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.107639 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 441/913] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:26.113178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4769ca-17dd-482a-afc0-8a076189f4eb None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.113609 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4769ca-17dd-482a-afc0-8a076189f4eb None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.113785 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4769ca-17dd-482a-afc0-8a076189f4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.135061 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-9f4769ca-17dd-482a-afc0-8a076189f4eb None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:26.136262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-9f4769ca-17dd-482a-afc0-8a076189f4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:26.206980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 473/914] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:26.229408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea1d82-3529-4767-85ac-fb7c85c0e68f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:26.349070 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea1d82-3529-4767-85ac-fb7c85c0e68f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:26.376905 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38239535-b152-44f6-8a07-5010c737fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:26.377714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38239535-b152-44f6-8a07-5010c737fcb2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.377999 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38239535-b152-44f6-8a07-5010c737fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.378443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38239535-b152-44f6-8a07-5010c737fcb2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.390492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38239535-b152-44f6-8a07-5010c737fcb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-38239535-b152-44f6-8a07-5010c737fcb2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:26.398687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:26.399503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:26.400234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 442/915] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:16:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:26.407270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4769ca-17dd-482a-afc0-8a076189f4eb None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:26.511436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4769ca-17dd-482a-afc0-8a076189f4eb None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:26.545643 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0c12c-7a46-4b19-82f5-5c0101588eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:26.546663 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0c12c-7a46-4b19-82f5-5c0101588eb8 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.547169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0c12c-7a46-4b19-82f5-5c0101588eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.547642 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0c12c-7a46-4b19-82f5-5c0101588eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.548586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0c12c-7a46-4b19-82f5-5c0101588eb8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:26.549111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0c12c-7a46-4b19-82f5-5c0101588eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:26.550677 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0c12c-7a46-4b19-82f5-5c0101588eb8 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:26.560473 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 474/916] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:16:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:26.566472 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38239535-b152-44f6-8a07-5010c737fcb2 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:26.660107 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ca33d-d247-4a64-923c-0bb36b5ce61e None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.660515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ca33d-d247-4a64-923c-0bb36b5ce61e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.661013 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ca33d-d247-4a64-923c-0bb36b5ce61e None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.662874 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 475/917] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:26.669379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33ba35-f60e-461e-946b-12c11d6bfd67 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.669990 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33ba35-f60e-461e-946b-12c11d6bfd67 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.670477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33ba35-f60e-461e-946b-12c11d6bfd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.673402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38239535-b152-44f6-8a07-5010c737fcb2 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:26.689455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-0c33ba35-f60e-461e-946b-12c11d6bfd67 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:26.691058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-0c33ba35-f60e-461e-946b-12c11d6bfd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:26.710048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:26.710840 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.711122 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.711394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.721625 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:26.723066 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:26.742043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:26.743068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:26.744283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:26.744283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:26.744283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:26.744283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:26.746122 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 443/918] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:26.751669 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:26.771604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 476/919] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:26.795583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33ba35-f60e-461e-946b-12c11d6bfd67 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:26.863526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195f66-36f8-424d-a2da-3096a0f9a823 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:26.897311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:26.898272 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.898698 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.899110 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.900031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:26.900543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:26.902097 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33ba35-f60e-461e-946b-12c11d6bfd67 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:26.902232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:26.915240 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 444/920] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:26.920825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:26.947045 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:26.948164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:26.948635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:26.949135 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:26.950066 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:26.951702 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:26.965931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:26.966862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:16:26.967586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:26.974727 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 477/921] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:26.976338 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.025893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964f9b3-73db-431f-9094-1fce0bb168a4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.059872 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.060898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.061225 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.061591 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.068819 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.070454 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.079049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65d71e-562e-4396-965d-a2039aa7a9b5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:27.086086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:27.086945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:27.087784 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:27.092326 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 445/922] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:16:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:27.093308 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.124689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5c964-3e23-46c9-9e5e-eacdc169c1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.125399 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5c964-3e23-46c9-9e5e-eacdc169c1fd None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.125684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5c964-3e23-46c9-9e5e-eacdc169c1fd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.125969 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5c964-3e23-46c9-9e5e-eacdc169c1fd None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.126689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5c964-3e23-46c9-9e5e-eacdc169c1fd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:27.126999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5c964-3e23-46c9-9e5e-eacdc169c1fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.128375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5c964-3e23-46c9-9e5e-eacdc169c1fd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.133289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 478/923] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:27.134516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39430abb-e195-4742-89a0-5956e9a6eeea None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.134797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39430abb-e195-4742-89a0-5956e9a6eeea None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.135072 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39430abb-e195-4742-89a0-5956e9a6eeea None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.136342 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 479/924] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:27.140213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39430abb-e195-4742-89a0-5956e9a6eeea None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.198137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6723bf-73e0-4ce7-8748-5d2ccf07ab07 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.234059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17926ea5-c2ff-41c7-a3b7-508d26e9b10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.234729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926ea5-c2ff-41c7-a3b7-508d26e9b10e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.235014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926ea5-c2ff-41c7-a3b7-508d26e9b10e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.235289 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17926ea5-c2ff-41c7-a3b7-508d26e9b10e None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.236120 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17926ea5-c2ff-41c7-a3b7-508d26e9b10e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:27.236430 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17926ea5-c2ff-41c7-a3b7-508d26e9b10e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.237880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17926ea5-c2ff-41c7-a3b7-508d26e9b10e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.240152 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39430abb-e195-4742-89a0-5956e9a6eeea None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.259249 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 446/925] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:27.260536 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56215d9b-2399-44c5-bbc9-d80eaeec2b3a None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.260816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56215d9b-2399-44c5-bbc9-d80eaeec2b3a None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.261111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56215d9b-2399-44c5-bbc9-d80eaeec2b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.278463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-56215d9b-2399-44c5-bbc9-d80eaeec2b3a None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:27.280269 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-56215d9b-2399-44c5-bbc9-d80eaeec2b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:27.286484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.287144 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.287421 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.287692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.289001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:27.289353 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.290713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.300028 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 480/926] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:27.300028 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.357669 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 447/927] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:27.362335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56215d9b-2399-44c5-bbc9-d80eaeec2b3a None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.399792 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7fb9-e383-4c0c-ae6f-e89cd7875210 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.435848 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.436648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.436931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.437234 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.450639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.453019 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.472152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56215d9b-2399-44c5-bbc9-d80eaeec2b3a None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:27.474506 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:27.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:27.476668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:27.476668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:27.476668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:27.476668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:27.482217 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 481/928] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:27.483106 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.511366 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda03657-2c3a-419a-8548-493d38ed4c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.512026 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda03657-2c3a-419a-8548-493d38ed4c30 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.512307 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda03657-2c3a-419a-8548-493d38ed4c30 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.512585 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda03657-2c3a-419a-8548-493d38ed4c30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.513410 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda03657-2c3a-419a-8548-493d38ed4c30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.514770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda03657-2c3a-419a-8548-493d38ed4c30 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.537089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 448/929] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:16:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:27.537089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda03657-2c3a-419a-8548-493d38ed4c30 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.619773 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe1462-95db-4b2b-a235-f65193ad7836 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.669232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.670544 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.671070 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.671560 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.672577 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.674483 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.677688 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda03657-2c3a-419a-8548-493d38ed4c30 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:27.688889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:27.690054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:27.690780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:27.693476 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 482/930] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:27.701162 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.727502 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.728226 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.728513 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.728794 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.729576 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:27.729888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.731333 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.746400 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 449/931] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:27.751973 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.831849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f34363-d3e8-4be4-ab14-d51e22c49d34 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.868569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.869262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.869548 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.869828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.870545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:27.870854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.872231 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.877745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 483/932] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:27.884419 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:27.892897 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af0ce8-4081-4a03-b248-e251553b85fd None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:27.934096 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a4147-79c5-48c7-90ce-1d1a877d8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:27.935929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4147-79c5-48c7-90ce-1d1a877d8a52 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.936215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4147-79c5-48c7-90ce-1d1a877d8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.936493 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a4147-79c5-48c7-90ce-1d1a877d8a52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.943729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a4147-79c5-48c7-90ce-1d1a877d8a52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:27.945515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a4147-79c5-48c7-90ce-1d1a877d8a52 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-325a4147-79c5-48c7-90ce-1d1a877d8a52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:27.961951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:27.962730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 450/933] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e46f7-26e9-4a8a-b1f8-39519ea816df None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:27.968771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e46f7-26e9-4a8a-b1f8-39519ea816df None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:27.969332 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e46f7-26e9-4a8a-b1f8-39519ea816df None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:27.969332 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 451/934] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:27.969332 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e46f7-26e9-4a8a-b1f8-39519ea816df None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:28.000432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb58c5-ab4a-4168-a500-e05059f2086f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.042354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e12d3-8106-4b22-99d8-8f789440a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.043421 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e12d3-8106-4b22-99d8-8f789440a275 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.043421 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e12d3-8106-4b22-99d8-8f789440a275 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.043610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e12d3-8106-4b22-99d8-8f789440a275 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.044281 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e12d3-8106-4b22-99d8-8f789440a275 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:28.044592 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e12d3-8106-4b22-99d8-8f789440a275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.046002 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e12d3-8106-4b22-99d8-8f789440a275 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:28.051186 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 484/935] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:28.052640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d8d4c-d468-4655-903a-7b63ef3698ee None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.052923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d8d4c-d468-4655-903a-7b63ef3698ee None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.053293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d8d4c-d468-4655-903a-7b63ef3698ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.074749 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-3e5d8d4c-d468-4655-903a-7b63ef3698ee None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:28.076502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-3e5d8d4c-d468-4655-903a-7b63ef3698ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:28.079790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e46f7-26e9-4a8a-b1f8-39519ea816df None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.115444 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.116797 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.117105 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.117390 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.118494 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:28.118760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.120190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:28.132796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 452/936] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:28.134012 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:28.157211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 485/937] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:28.158046 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d8d4c-d468-4655-903a-7b63ef3698ee None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:28.240763 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4bfc4-6b90-45a8-ab79-f7e13f7c9a70 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.270021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d8d4c-d468-4655-903a-7b63ef3698ee None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.280014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.280676 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.280953 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.281252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.282031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] RBAC: Adding query filter params (interface=public, service_id=84e6e19939f844eab8788c9ec94f6c0e, region_id=RegionOne) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:28.282337 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.283684 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:28.293382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 453/938] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:16:28 2022] GET /identity/v3/endpoints?service_id=84e6e19939f844eab8788c9ec94f6c0e&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:28.293382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:28.304115 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe77356-02b8-425b-820e-a30305e9b0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.305074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe77356-02b8-425b-820e-a30305e9b0fa None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.305357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe77356-02b8-425b-820e-a30305e9b0fa None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.305636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe77356-02b8-425b-820e-a30305e9b0fa None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/e28363f2360842958bfe92394afe9651/roles/5bdec21d519c4a929fc1ddde15a7b9ca` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.307915 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe77356-02b8-425b-820e-a30305e9b0fa None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=e28363f2360842958bfe92394afe9651, role_id=5bdec21d519c4a929fc1ddde15a7b9ca)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.309393 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe77356-02b8-425b-820e-a30305e9b0fa None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:28.324615 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 486/939] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:16:28 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/e28363f2360842958bfe92394afe9651/roles/5bdec21d519c4a929fc1ddde15a7b9ca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:16:28.325474 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe77356-02b8-425b-820e-a30305e9b0fa None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:28.434719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa91aab-4337-40a3-9564-b86d93b89682 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.452253 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe77356-02b8-425b-820e-a30305e9b0fa None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.481781 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.483608 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.484435 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.484861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.485850 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.488172 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:28.491421 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afac7e1-468e-49ef-8db9-25f80c239f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.492141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afac7e1-468e-49ef-8db9-25f80c239f3d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.492491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afac7e1-468e-49ef-8db9-25f80c239f3d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.492840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afac7e1-468e-49ef-8db9-25f80c239f3d None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.493937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afac7e1-468e-49ef-8db9-25f80c239f3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.495420 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afac7e1-468e-49ef-8db9-25f80c239f3d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:28.502547 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 487/940] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:16:28 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:28.504377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:28.505244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:28.506011 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 454/941] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:16:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:28.512953 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afac7e1-468e-49ef-8db9-25f80c239f3d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:28.635391 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afac7e1-468e-49ef-8db9-25f80c239f3d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.671938 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.672619 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.672897 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.673196 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.673979 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:28.674288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.675720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:28.683013 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 488/942] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:28.688832 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:28.794237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742df46-42ac-42d0-b8b5-5d89ce5f16a8 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.828470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bf0f5-634c-49af-b0be-2f2d7a53b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.829160 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bf0f5-634c-49af-b0be-2f2d7a53b938 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.829441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bf0f5-634c-49af-b0be-2f2d7a53b938 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.829713 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bf0f5-634c-49af-b0be-2f2d7a53b938 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.830417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bf0f5-634c-49af-b0be-2f2d7a53b938 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:28.830719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bf0f5-634c-49af-b0be-2f2d7a53b938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.832071 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bf0f5-634c-49af-b0be-2f2d7a53b938 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:28.837899 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 455/943] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:28.843479 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:28.949624 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81668476-069f-4edf-9b34-e01698e2d33f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:28.983604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89f608-0048-4785-8241-45a637da1028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:28.984265 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89f608-0048-4785-8241-45a637da1028 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:28.984549 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89f608-0048-4785-8241-45a637da1028 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:28.984825 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89f608-0048-4785-8241-45a637da1028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:28.985607 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89f608-0048-4785-8241-45a637da1028 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:28.986976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89f608-0048-4785-8241-45a637da1028 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:29.010812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 489/944] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:16:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:29.944856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e79a5-05e6-40de-9aa7-303eee667ff7 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:29.945402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e79a5-05e6-40de-9aa7-303eee667ff7 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:29.945751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e79a5-05e6-40de-9aa7-303eee667ff7 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:29.947544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 456/945] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:29.953011 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6351d-9e14-43bb-ac23-1885f33d16f3 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:29.953428 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6351d-9e14-43bb-ac23-1885f33d16f3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:29.953785 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6351d-9e14-43bb-ac23-1885f33d16f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:29.971066 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-b7a6351d-9e14-43bb-ac23-1885f33d16f3 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:29.972295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-b7a6351d-9e14-43bb-ac23-1885f33d16f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:30.049203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 490/946] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:30.068702 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e79a5-05e6-40de-9aa7-303eee667ff7 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:30.176155 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e79a5-05e6-40de-9aa7-303eee667ff7 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:30.210765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:30.211353 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:30.211637 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:30.211914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:30.212633 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:30.214190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:30.215767 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: WARNING keystone.server.flask.common [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 15:16:30.238247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.sql.core [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:16:30.238247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 15:16:30.238247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [parameters: {'id': '3c7cff39ae324b59b3ca29a0938bab7a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 15:16:30.238247 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=143116) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3c7cff39ae324b59b3ca29a0938bab7a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3c7cff39ae324b59b3ca29a0938bab7a. May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application conn.query(q) May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:16:30.246236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application result.read() May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 09 15:16:30.247005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return project_ref.to_dict() May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application next(self.gen) May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application yield resource May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application next(self.gen) May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise self.value May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application session.commit() May 09 15:16:30.247929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.session.flush() May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application compat.raise_( May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:16:30.248724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:16:30.250192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application conn.query(q) May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application result.read() May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:16:30.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [parameters: {'id': '3c7cff39ae324b59b3ca29a0938bab7a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:30.251729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:30.252419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.Conflict( May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3c7cff39ae324b59b3ca29a0938bab7a. May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:30.257549 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 457/947] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 9 15:16:30 2022] POST /identity/v3/projects => generated 262 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:30.258810 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6351d-9e14-43bb-ac23-1885f33d16f3 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:30.358793 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6351d-9e14-43bb-ac23-1885f33d16f3 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:30.393680 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9a61-552c-44c1-a6f4-504a80aba8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:30.394476 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9a61-552c-44c1-a6f4-504a80aba8e9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:30.394758 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9a61-552c-44c1-a6f4-504a80aba8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:30.395032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9a61-552c-44c1-a6f4-504a80aba8e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:30.402504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9a61-552c-44c1-a6f4-504a80aba8e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:30.403959 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9a61-552c-44c1-a6f4-504a80aba8e9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d54b9a61-552c-44c1-a6f4-504a80aba8e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:30.417961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:30.419891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:30.421971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:30.421971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:30.421971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:30.421971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:30.421971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:30.421971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:30.421971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:30.421971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:30.426467 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 491/948] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:30.426651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:30.530297 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d6183-15e3-4958-b704-8a1b379aa56e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:30.566853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2416d8-7f9c-4618-bf67-cedbe5d345cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:30.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2416d8-7f9c-4618-bf67-cedbe5d345cc None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:30.567824 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2416d8-7f9c-4618-bf67-cedbe5d345cc None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:30.568104 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2416d8-7f9c-4618-bf67-cedbe5d345cc None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:30.568884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2416d8-7f9c-4618-bf67-cedbe5d345cc None admin] RBAC: Adding query filter params (name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:30.569443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2416d8-7f9c-4618-bf67-cedbe5d345cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:30.571148 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2416d8-7f9c-4618-bf67-cedbe5d345cc None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:30.602642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 458/949] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:30.755514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1b67f-cd61-467a-bb2c-0d6908d60ac5 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:30.755886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1b67f-cd61-467a-bb2c-0d6908d60ac5 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:30.756060 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1b67f-cd61-467a-bb2c-0d6908d60ac5 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:30.757896 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 492/950] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:30.763815 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42d9d6-d277-4b1b-b94c-66c1b4b8489d None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:30.764086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42d9d6-d277-4b1b-b94c-66c1b4b8489d None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:30.764368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42d9d6-d277-4b1b-b94c-66c1b4b8489d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:30.784208 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-cd42d9d6-d277-4b1b-b94c-66c1b4b8489d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:30.785703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-cd42d9d6-d277-4b1b-b94c-66c1b4b8489d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:30.819606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eff72d-2a9e-4f27-bcae-882e43e955b0 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:30.819606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eff72d-2a9e-4f27-bcae-882e43e955b0 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:30.819606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eff72d-2a9e-4f27-bcae-882e43e955b0 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:30.819606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 493/951] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:30.826148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb0f3-ecd4-4509-b8de-8db0e093b3da None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:30.826437 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb0f3-ecd4-4509-b8de-8db0e093b3da None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:30.826715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb0f3-ecd4-4509-b8de-8db0e093b3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:30.847093 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-8a5cb0f3-ecd4-4509-b8de-8db0e093b3da None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:30.848329 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-8a5cb0f3-ecd4-4509-b8de-8db0e093b3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:30.868720 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 459/952] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:30.893409 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42d9d6-d277-4b1b-b94c-66c1b4b8489d None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:30.936945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 494/953] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:30.960682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cb0f3-ecd4-4509-b8de-8db0e093b3da None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.007671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42d9d6-d277-4b1b-b94c-66c1b4b8489d None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.042450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b5de6-6487-4d6a-939d-65a7e12931e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.043181 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b5de6-6487-4d6a-939d-65a7e12931e0 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.043461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b5de6-6487-4d6a-939d-65a7e12931e0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.043733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b5de6-6487-4d6a-939d-65a7e12931e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.057679 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b5de6-6487-4d6a-939d-65a7e12931e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:31.059183 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cb0f3-ecd4-4509-b8de-8db0e093b3da None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-2f0b5de6-6487-4d6a-939d-65a7e12931e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:31.067446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:31.068301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:31.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:31.070158 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 460/954] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:16:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:31.076521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b5de6-6487-4d6a-939d-65a7e12931e0 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.095550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.096522 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.097140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.097418 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.098082 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.099886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:31.116156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:31.119108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:31.121004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:31.122767 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 495/955] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:31.125907 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.197312 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b5de6-6487-4d6a-939d-65a7e12931e0 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.242797 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.244360 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.244847 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.245392 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.245651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c0bf5-e094-4adb-b430-18585a51612d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.246561 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:31.247250 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.249671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.260664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 461/956] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:31.267243 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.288930 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.289661 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.289910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.290179 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.290882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:31.291181 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.292585 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.297802 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 496/957] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:31.303279 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.390787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de58e9d-b4ed-46e2-8c2d-0c6e5ca4a0f2 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.402449 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a78d2f-2292-40ce-a224-b296f86a06c2 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.424298 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbf3ef-f7d7-43cf-b4ec-c70e0fc8a29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.425114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf3ef-f7d7-43cf-b4ec-c70e0fc8a29a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.425398 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf3ef-f7d7-43cf-b4ec-c70e0fc8a29a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.425681 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf3ef-f7d7-43cf-b4ec-c70e0fc8a29a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.435534 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e05cfd-0b41-4648-8511-33c9a01dbce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.436196 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05cfd-0b41-4648-8511-33c9a01dbce1 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.436478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05cfd-0b41-4648-8511-33c9a01dbce1 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.436775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05cfd-0b41-4648-8511-33c9a01dbce1 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.437571 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e05cfd-0b41-4648-8511-33c9a01dbce1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:31.437876 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e05cfd-0b41-4648-8511-33c9a01dbce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.439155 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf3ef-f7d7-43cf-b4ec-c70e0fc8a29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.439260 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e05cfd-0b41-4648-8511-33c9a01dbce1 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.440852 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf3ef-f7d7-43cf-b4ec-c70e0fc8a29a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.443973 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 497/958] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:31.445311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2ded1-b057-486e-addb-4160f63e5055 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.445596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2ded1-b057-486e-addb-4160f63e5055 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.445876 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2ded1-b057-486e-addb-4160f63e5055 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.451126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 498/959] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:31.451126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2ded1-b057-486e-addb-4160f63e5055 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-9dbbf3ef-f7d7-43cf-b4ec-c70e0fc8a29a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:31.464531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:31.465382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:31.471084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:31.471084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:31.471084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:31.471084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:31.471084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 462/960] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:31.471084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49d368-e722-4243-96d2-e735f789baeb None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.471084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49d368-e722-4243-96d2-e735f789baeb None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.471084 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49d368-e722-4243-96d2-e735f789baeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.484936 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-5a49d368-e722-4243-96d2-e735f789baeb None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:31.486229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-5a49d368-e722-4243-96d2-e735f789baeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:31.553499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 463/961] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:31.554270 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49d368-e722-4243-96d2-e735f789baeb None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.566692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2ded1-b057-486e-addb-4160f63e5055 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.603842 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a7f9d-aeda-4f59-8de6-d52241d7b862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.604504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7f9d-aeda-4f59-8de6-d52241d7b862 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.604788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7f9d-aeda-4f59-8de6-d52241d7b862 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.605085 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a7f9d-aeda-4f59-8de6-d52241d7b862 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.605837 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a7f9d-aeda-4f59-8de6-d52241d7b862 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.607194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a7f9d-aeda-4f59-8de6-d52241d7b862 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.626001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 499/962] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:16:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:16:31.626903 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a7f9d-aeda-4f59-8de6-d52241d7b862 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.659389 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49d368-e722-4243-96d2-e735f789baeb None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.719545 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.720326 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.720681 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.721028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.721868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:31.722246 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.724386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.745926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 464/963] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:31.747220 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.757116 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a7f9d-aeda-4f59-8de6-d52241d7b862 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.795175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.796348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.796852 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.797348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] PATH_INFO: `/v3/services/compute` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.798242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.799941 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:31.817218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:31.818788 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:31.820629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:31.825596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 500/964] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:31.833792 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.856825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d6cb7-7dc1-401c-9620-e73c1e8f4457 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.892809 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.893666 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.893964 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.894227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:31.905158 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:31.906603 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:31.921450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:31.922532 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:31.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 465/965] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:31.928668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:31.959501 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c5604-8d70-4020-8954-5d1f7fea5e5d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:31.997074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:31.998421 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:31.999115 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:31.999568 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:32.000424 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] RBAC: Adding query filter params (name=compute) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:32.001088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:32.003547 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:32.016901 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 501/966] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:32.018064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:32.040111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f34d65-a896-4fe2-b0ce-f71536fea750 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:32.074414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:32.075107 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:32.075284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:32.075764 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:32.076618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:32.076937 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:32.078582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:32.096956 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 466/967] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:16:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:32.102264 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:32.124503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47fd68-a224-4f3f-af14-2a5534b6e284 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:32.163099 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:32.164056 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:32.164514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:32.164963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:32.165891 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] RBAC: Adding query filter params (type=compute) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:32.166433 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:32.167959 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:32.172603 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 502/968] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:32.210514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bf5d9-525c-4e9f-88a2-fbda1fea0a34 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:32.247530 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593272-6085-402e-8701-0e7f511b37ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:32.248197 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593272-6085-402e-8701-0e7f511b37ff None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:32.257526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593272-6085-402e-8701-0e7f511b37ff None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:32.257859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593272-6085-402e-8701-0e7f511b37ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:32.349719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48593272-6085-402e-8701-0e7f511b37ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:32.351472 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48593272-6085-402e-8701-0e7f511b37ff None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:32.451745 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 467/969] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:16:32.457621 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:32.561918 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98191d0-97e8-4137-973f-7809a021b248 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:32.595999 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:32.596793 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:32.597094 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:32.597374 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:32.608871 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:32.610276 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:32.628136 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:32.628937 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:32.629775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:32.629775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:32.629775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:32.629775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:32.629775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 503/970] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:32.636045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593272-6085-402e-8701-0e7f511b37ff None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:32.740336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593272-6085-402e-8701-0e7f511b37ff None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:32.779302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:32.780294 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:32.780754 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:32.781251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:32.782151 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:32.782641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:32.784174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:32.796560 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 468/971] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:32.802248 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:32.917800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287d923-c4e8-40b3-b36a-4cb502b27f60 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:32.954273 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5c125-6a77-4349-bfaa-9162a529a803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:32.955073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5c125-6a77-4349-bfaa-9162a529a803 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:32.955351 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5c125-6a77-4349-bfaa-9162a529a803 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:32.955623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5c125-6a77-4349-bfaa-9162a529a803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:32.962598 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5c125-6a77-4349-bfaa-9162a529a803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:32.964101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5c125-6a77-4349-bfaa-9162a529a803 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-b5e5c125-6a77-4349-bfaa-9162a529a803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:32.978211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:32.979014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:32.979797 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 504/972] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:32.986582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:33.074231 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b38c4d-059e-4162-99b0-49c680b61052 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:33.074738 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b38c4d-059e-4162-99b0-49c680b61052 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:33.075041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b38c4d-059e-4162-99b0-49c680b61052 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:33.077514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 505/973] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:33.087264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a001c-9662-4eb6-97ac-afd3f5623415 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:33.087550 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a001c-9662-4eb6-97ac-afd3f5623415 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:33.087831 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a001c-9662-4eb6-97ac-afd3f5623415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:33.098358 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec16bc7-c708-40c3-b0c1-67b70c5ccd92 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:33.108805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-b87a001c-9662-4eb6-97ac-afd3f5623415 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:33.110439 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-b87a001c-9662-4eb6-97ac-afd3f5623415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:33.135134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:33.136007 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:33.136436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:33.136855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:33.137812 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:33.138285 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:33.139855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:33.152638 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 469/974] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:33.158691 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:33.187652 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 506/975] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:33.214470 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a001c-9662-4eb6-97ac-afd3f5623415 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:33.269168 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21799-8842-4e12-b15c-28e4e782f1fb None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:33.303902 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:33.304418 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:33.304706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:33.304987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:33.305861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] RBAC: Adding query filter params (role.id=5bdec21d519c4a929fc1ddde15a7b9ca, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=e28363f2360842958bfe92394afe9651) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:33.306177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:33.307611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:33.313913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 470/976] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:33 2022] GET /identity/v3/role_assignments?user.id=e28363f2360842958bfe92394afe9651&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=5bdec21d519c4a929fc1ddde15a7b9ca => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:33.316727 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a001c-9662-4eb6-97ac-afd3f5623415 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:33.357012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:33.357981 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:33.358409 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:33.358853 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:33.359718 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:33.361380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:33.363120 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: WARNING keystone.server.flask.common [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 15:16:33.377721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.sql.core [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:16:33.377721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 15:16:33.377721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [parameters: {'user_id': 'dff183dc65da464590c83608410ec0f5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 15:16:33.377721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=143117) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application conn.query(q) May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application result.read() May 09 15:16:33.396138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:16:33.397236 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application next(self.gen) May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application yield resource May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application next(self.gen) May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise self.value May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application session.commit() May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:16:33.398379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self.session.flush() May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application compat.raise_( May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:16:33.399343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application result = connection._execute_20( May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:16:33.400257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application conn.query(q) May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application result.read() May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:16:33.401000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dff183dc65da464590c83608410ec0f5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:33.401786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 09 15:16:33.402502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 09 15:16:33.403284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 09 15:16:33.403284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 09 15:16:33.403284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:16:33.403284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:16:33.403284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 15:16:33.403284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:33.405669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 507/977] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:16:33 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:33.411725 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:33.525483 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769cd38-bb5f-4517-9210-a41edaabb468 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:33.560557 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d520abb-f79a-42ac-ae79-5eb3d1cb6819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:33.561448 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d520abb-f79a-42ac-ae79-5eb3d1cb6819 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:33.561735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d520abb-f79a-42ac-ae79-5eb3d1cb6819 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:33.562067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d520abb-f79a-42ac-ae79-5eb3d1cb6819 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:33.569981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d520abb-f79a-42ac-ae79-5eb3d1cb6819 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:33.571436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d520abb-f79a-42ac-ae79-5eb3d1cb6819 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-6d520abb-f79a-42ac-ae79-5eb3d1cb6819 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:33.586142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:33.587017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:33.587842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 471/978] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:33.595019 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:33.703987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ca6e4-315b-445c-b928-3ed5989767f2 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:33.738993 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f446220-54b7-40fe-8ae2-3e0a6ff66ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:33.740018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f446220-54b7-40fe-8ae2-3e0a6ff66ed6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:33.740504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f446220-54b7-40fe-8ae2-3e0a6ff66ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:33.741062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f446220-54b7-40fe-8ae2-3e0a6ff66ed6 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:33.742074 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f446220-54b7-40fe-8ae2-3e0a6ff66ed6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:33.742582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f446220-54b7-40fe-8ae2-3e0a6ff66ed6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:33.744672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f446220-54b7-40fe-8ae2-3e0a6ff66ed6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:33.783996 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 508/979] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:16:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:34.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5426c04-67d7-43f7-8745-875a2d40d4aa None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.127329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5426c04-67d7-43f7-8745-875a2d40d4aa None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.127329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5426c04-67d7-43f7-8745-875a2d40d4aa None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.128843 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 472/980] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:34.134684 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eabad-a4ba-4ba3-a8bc-a5c9f9ddf712 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.135102 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eabad-a4ba-4ba3-a8bc-a5c9f9ddf712 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.135264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eabad-a4ba-4ba3-a8bc-a5c9f9ddf712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.155318 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-a42eabad-a4ba-4ba3-a8bc-a5c9f9ddf712 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:34.156834 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-a42eabad-a4ba-4ba3-a8bc-a5c9f9ddf712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:34.222431 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 509/981] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:34.245013 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5426c04-67d7-43f7-8745-875a2d40d4aa None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:34.353871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5426c04-67d7-43f7-8745-875a2d40d4aa None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:34.387787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8614737-e16f-415c-9a93-b4f86032f0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:34.388520 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614737-e16f-415c-9a93-b4f86032f0a8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.388798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614737-e16f-415c-9a93-b4f86032f0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.389086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8614737-e16f-415c-9a93-b4f86032f0a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.389739 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614737-e16f-415c-9a93-b4f86032f0a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:34.391097 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8614737-e16f-415c-9a93-b4f86032f0a8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-e8614737-e16f-415c-9a93-b4f86032f0a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:34.403414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:34.404373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:34.405174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:34.406599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 473/982] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:34.410673 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eabad-a4ba-4ba3-a8bc-a5c9f9ddf712 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:34.505175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eabad-a4ba-4ba3-a8bc-a5c9f9ddf712 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:34.524813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805c35-a92a-477c-b685-57afd9d65d12 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.525132 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805c35-a92a-477c-b685-57afd9d65d12 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.525412 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805c35-a92a-477c-b685-57afd9d65d12 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.526794 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 474/983] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:34.531739 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228bc07-eb50-49c5-97f0-20298689e582 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.532023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228bc07-eb50-49c5-97f0-20298689e582 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.532301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228bc07-eb50-49c5-97f0-20298689e582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.540698 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:34.541655 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.542100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.542516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.543364 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:34.543826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:34.546098 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:34.549708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-b228bc07-eb50-49c5-97f0-20298689e582 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:34.550915 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-b228bc07-eb50-49c5-97f0-20298689e582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:34.552128 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 510/984] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:34.558327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:34.625102 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 475/985] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:34.645691 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228bc07-eb50-49c5-97f0-20298689e582 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:34.656448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51517328-b37f-4726-847e-509d6eb1ae02 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:34.690374 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:34.691260 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.691713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.692128 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.692974 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:34.693447 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:34.695518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:34.700570 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 511/986] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:34.706555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:34.740713 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228bc07-eb50-49c5-97f0-20298689e582 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:34.776179 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0411843-8f67-4959-8deb-c2d09318a6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:34.777075 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0411843-8f67-4959-8deb-c2d09318a6f5 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.777527 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0411843-8f67-4959-8deb-c2d09318a6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.777940 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0411843-8f67-4959-8deb-c2d09318a6f5 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.778795 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0411843-8f67-4959-8deb-c2d09318a6f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:34.780286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0411843-8f67-4959-8deb-c2d09318a6f5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:34.794583 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 476/987] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:16:34 2022] POST /identity/v3/services => generated 242 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:16:34.804779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b03f8-7b1b-40a0-aa22-1d914b9d73fb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:34.849830 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fc6c4-9b4f-443e-948e-dffb0ae35f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:34.851064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fc6c4-9b4f-443e-948e-dffb0ae35f61 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:34.851343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fc6c4-9b4f-443e-948e-dffb0ae35f61 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:34.851613 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fc6c4-9b4f-443e-948e-dffb0ae35f61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:34.852414 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fc6c4-9b4f-443e-948e-dffb0ae35f61 None admin] RBAC: Adding query filter params (interface=admin, service_id=84e6e19939f844eab8788c9ec94f6c0e, region_id=RegionOne) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:34.852722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fc6c4-9b4f-443e-948e-dffb0ae35f61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:34.854172 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fc6c4-9b4f-443e-948e-dffb0ae35f61 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:34.860391 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 512/988] 10.222.0.13 () {56 vars in 1249 bytes} [Mon May 9 15:16:34 2022] GET /identity/v3/endpoints?service_id=84e6e19939f844eab8788c9ec94f6c0e&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:34.866724 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0411843-8f67-4959-8deb-c2d09318a6f5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:34.971547 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0411843-8f67-4959-8deb-c2d09318a6f5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:35.007039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525c509-f772-4fb9-927c-31fccbf12998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:35.008091 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525c509-f772-4fb9-927c-31fccbf12998 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:35.008592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525c509-f772-4fb9-927c-31fccbf12998 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:35.009021 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525c509-f772-4fb9-927c-31fccbf12998 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:35.009796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525c509-f772-4fb9-927c-31fccbf12998 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:35.011166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525c509-f772-4fb9-927c-31fccbf12998 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:35.017860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 477/989] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:16:34 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:16:35.689408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2133577-819a-4c2f-8930-02238e983c8e None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:35.689914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2133577-819a-4c2f-8930-02238e983c8e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:35.689986 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2133577-819a-4c2f-8930-02238e983c8e None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:35.691489 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 513/990] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:35.696629 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f3241-8753-41b5-a4eb-7a82e6989e10 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:35.697001 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f3241-8753-41b5-a4eb-7a82e6989e10 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:35.697228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f3241-8753-41b5-a4eb-7a82e6989e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:35.713625 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-aa1f3241-8753-41b5-a4eb-7a82e6989e10 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:35.714807 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-aa1f3241-8753-41b5-a4eb-7a82e6989e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:35.778817 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 478/991] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:35.800861 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2133577-819a-4c2f-8930-02238e983c8e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:35.898589 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2133577-819a-4c2f-8930-02238e983c8e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:35.931622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:35.932421 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:35.932704 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:35.932981 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] PATH_INFO: `/v3/services/image` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:35.933675 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:35.935076 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:35.947651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:35.948610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:35.949987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:35.951037 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 514/992] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:16:35 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:35.954924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f3241-8753-41b5-a4eb-7a82e6989e10 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:36.050461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f3241-8753-41b5-a4eb-7a82e6989e10 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:36.082482 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b8dc-a569-40fc-8aab-b679e8339c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:36.083155 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b8dc-a569-40fc-8aab-b679e8339c6d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:36.083436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b8dc-a569-40fc-8aab-b679e8339c6d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:36.083709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b8dc-a569-40fc-8aab-b679e8339c6d None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:36.084407 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424b8dc-a569-40fc-8aab-b679e8339c6d None admin] RBAC: Adding query filter params (name=image) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:36.085275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424b8dc-a569-40fc-8aab-b679e8339c6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:36.086624 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424b8dc-a569-40fc-8aab-b679e8339c6d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:36.095976 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 479/993] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:36.096931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:36.196024 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e284277-e27b-4503-9406-1d2b479fb724 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:36.228925 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af1b9d-3e24-4de7-99ad-745b59272000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:36.229546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af1b9d-3e24-4de7-99ad-745b59272000 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:36.229826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af1b9d-3e24-4de7-99ad-745b59272000 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:36.230099 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af1b9d-3e24-4de7-99ad-745b59272000 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:36.230805 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af1b9d-3e24-4de7-99ad-745b59272000 None admin] RBAC: Adding query filter params (type=image) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:36.231110 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af1b9d-3e24-4de7-99ad-745b59272000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:36.232462 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af1b9d-3e24-4de7-99ad-745b59272000 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:36.236924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 515/994] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:37.061796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995713-b694-4657-a55e-dcd7f8feff60 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.062317 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995713-b694-4657-a55e-dcd7f8feff60 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.062390 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995713-b694-4657-a55e-dcd7f8feff60 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.063914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 480/995] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:37.070174 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf83025-46e0-475a-b4ba-e0eafca6ff91 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.070697 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf83025-46e0-475a-b4ba-e0eafca6ff91 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.070769 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf83025-46e0-475a-b4ba-e0eafca6ff91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.088348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-caf83025-46e0-475a-b4ba-e0eafca6ff91 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:37.089616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-caf83025-46e0-475a-b4ba-e0eafca6ff91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:37.155406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 516/996] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:37.178152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb995713-b694-4657-a55e-dcd7f8feff60 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.265239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d41a3-af46-42ee-b832-220fd4dc5234 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.265528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d41a3-af46-42ee-b832-220fd4dc5234 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.265803 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d41a3-af46-42ee-b832-220fd4dc5234 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.267333 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 517/997] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:37.272178 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee09ed2-cb7f-4dd8-9cd4-db5a4a67e3de None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.272466 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee09ed2-cb7f-4dd8-9cd4-db5a4a67e3de None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.272745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee09ed2-cb7f-4dd8-9cd4-db5a4a67e3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.287195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb995713-b694-4657-a55e-dcd7f8feff60 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.289575 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-8ee09ed2-cb7f-4dd8-9cd4-db5a4a67e3de None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:37.290759 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-8ee09ed2-cb7f-4dd8-9cd4-db5a4a67e3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:37.322189 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.322903 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.323182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.323451 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] PATH_INFO: `/v3/services/compute` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.324089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:37.325483 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:37.337108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:37.338286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:37.339059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 481/998] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:37.344914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.369382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 518/999] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:37.385657 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee09ed2-cb7f-4dd8-9cd4-db5a4a67e3de None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.448009 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0f054-77e6-47bc-9cbe-32debcac7066 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.483526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.484203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.484482 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.484759 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.485498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:37.485811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:37.487274 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:37.492461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee09ed2-cb7f-4dd8-9cd4-db5a4a67e3de None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.499067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 482/1000] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:37.500419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.529338 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.530144 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.530430 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.530708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.531362 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:37.532734 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:37.545180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:37.546058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:37.546811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 519/1001] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:16:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:37.552296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.604323 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7beab-6b8c-43c2-9a30-5aaea1c73cb4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.636515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.637218 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.637504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.637783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.638485 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:37.638791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:37.640148 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:37.645024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 483/1002] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:37.651139 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.651597 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1a62-c18d-4027-b755-51ec90aab0a4 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.685666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.686328 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.686664 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.686938 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.687695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:37.688006 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:37.689406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:37.699232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 520/1003] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:37.700387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.757959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa0f1e-78bf-4024-afac-fb294c3806f5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.794020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.794678 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.794962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.795228 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.796021 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] RBAC: Adding query filter params (interface=public, service_id=17607b59e7e344c9ab8aeaee2965bd22, region_id=RegionOne) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:37.796499 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:37.797733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:37.799095 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a5384-9b89-4e74-a5ff-246da3d8f849 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.802838 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 484/1004] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:16:37 2022] GET /identity/v3/endpoints?service_id=17607b59e7e344c9ab8aeaee2965bd22&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:37.808521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.834027 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.834700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.834987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.835265 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.835982 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:37.836294 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:37.837689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:37.842530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 521/1005] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:16:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:37.848041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:37.909133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bed21-cfd1-46f6-a9c4-cbf31f3f126e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.949032 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b4f0f-0166-4ab9-b269-ea2d7cb5e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.949729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b4f0f-0166-4ab9-b269-ea2d7cb5e84e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.950013 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b4f0f-0166-4ab9-b269-ea2d7cb5e84e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.950288 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b4f0f-0166-4ab9-b269-ea2d7cb5e84e None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.950968 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b4f0f-0166-4ab9-b269-ea2d7cb5e84e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:37.952341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b4f0f-0166-4ab9-b269-ea2d7cb5e84e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:37.956119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b6a7c-05eb-4d7a-8cfa-c7e7deb8a3fe None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:37.958463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 485/1006] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:16:37 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:16:37.997692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b36026-0af4-4e79-9e91-8b44181fbf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:37.998368 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b36026-0af4-4e79-9e91-8b44181fbf7e None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:37.998650 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b36026-0af4-4e79-9e91-8b44181fbf7e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:37.998924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b36026-0af4-4e79-9e91-8b44181fbf7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:37.999683 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b36026-0af4-4e79-9e91-8b44181fbf7e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:38.001108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b36026-0af4-4e79-9e91-8b44181fbf7e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:38.025092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 522/1007] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:16:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:38.491995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb011bb-250b-4263-876f-21f05f089439 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:38.492493 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb011bb-250b-4263-876f-21f05f089439 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:38.492562 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb011bb-250b-4263-876f-21f05f089439 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:38.494238 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 486/1008] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:38.499405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cf30-0ecd-47c4-86e2-1e68d9667fd2 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:38.499804 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cf30-0ecd-47c4-86e2-1e68d9667fd2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:38.499979 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1cf30-0ecd-47c4-86e2-1e68d9667fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:38.516975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-3ae1cf30-0ecd-47c4-86e2-1e68d9667fd2 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:38.518110 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-3ae1cf30-0ecd-47c4-86e2-1e68d9667fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:38.587302 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 523/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:38.609762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb011bb-250b-4263-876f-21f05f089439 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:38.710481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb011bb-250b-4263-876f-21f05f089439 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:38.742421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656a2ee-d9d6-4b3b-a21c-42a35993f6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:38.743345 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a2ee-d9d6-4b3b-a21c-42a35993f6f1 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:38.743623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a2ee-d9d6-4b3b-a21c-42a35993f6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:38.743894 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656a2ee-d9d6-4b3b-a21c-42a35993f6f1 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:38.744620 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656a2ee-d9d6-4b3b-a21c-42a35993f6f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:38.746003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656a2ee-d9d6-4b3b-a21c-42a35993f6f1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:38.765070 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 487/1010] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:16:38 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:40.393900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecb2b8-11b7-4516-9886-8042f2f1907a None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:40.394404 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecb2b8-11b7-4516-9886-8042f2f1907a None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:40.394475 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecb2b8-11b7-4516-9886-8042f2f1907a None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:40.395835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 524/1011] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:40.402002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3258425-5939-4fa5-ba81-2195fdda3032 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:40.402399 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3258425-5939-4fa5-ba81-2195fdda3032 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:40.402556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3258425-5939-4fa5-ba81-2195fdda3032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:40.419385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-f3258425-5939-4fa5-ba81-2195fdda3032 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:40.420874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-f3258425-5939-4fa5-ba81-2195fdda3032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:40.491518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 488/1012] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:40 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:40.514971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecb2b8-11b7-4516-9886-8042f2f1907a None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:40.614669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecb2b8-11b7-4516-9886-8042f2f1907a None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:40.648404 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:40.649260 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:40.649543 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:40.649817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] PATH_INFO: `/v3/services/compute` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:40.650493 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:40.651898 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:40.664820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:40.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:40.672168 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 525/1013] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:40.672881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3258425-5939-4fa5-ba81-2195fdda3032 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:40.770317 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3258425-5939-4fa5-ba81-2195fdda3032 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:40.802775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ec627-e645-41d3-b303-bf7e2c3abf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:40.803974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ec627-e645-41d3-b303-bf7e2c3abf62 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:40.804254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ec627-e645-41d3-b303-bf7e2c3abf62 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:40.804525 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ec627-e645-41d3-b303-bf7e2c3abf62 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:40.805253 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ec627-e645-41d3-b303-bf7e2c3abf62 None admin] RBAC: Adding query filter params (name=compute) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:40.805561 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ec627-e645-41d3-b303-bf7e2c3abf62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:40.806993 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ec627-e645-41d3-b303-bf7e2c3abf62 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:40.817031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 489/1014] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:40.818235 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:40.884700 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fe5d7-4309-4bab-98a8-afef64f71d87 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:40.885092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fe5d7-4309-4bab-98a8-afef64f71d87 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:40.885284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fe5d7-4309-4bab-98a8-afef64f71d87 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:40.886671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 490/1015] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:40.891699 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e9b7-a478-4d8b-be26-888c6b87bc66 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:40.891984 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e9b7-a478-4d8b-be26-888c6b87bc66 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:40.892258 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e9b7-a478-4d8b-be26-888c6b87bc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:40.908619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-2dd7e9b7-a478-4d8b-be26-888c6b87bc66 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:40.909821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-2dd7e9b7-a478-4d8b-be26-888c6b87bc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:40.921014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c471e1-4673-40c6-ad5f-d6baff75c472 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:40.958012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:40.958676 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:40.958962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:40.959242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:40.959983 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] RBAC: Adding query filter params (type=compute) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:40.960291 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:40.961811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:40.967285 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 526/1016] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:16:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:40.973357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:40.989389 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 491/1017] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:40 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:41.009600 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7e9b7-a478-4d8b-be26-888c6b87bc66 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:41.072767 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b5b2-9275-445a-a497-9be1a288267e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:41.102110 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7e9b7-a478-4d8b-be26-888c6b87bc66 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:41.104809 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058323e-6ca8-4f1a-bce7-59d2cfa1a24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:41.105484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058323e-6ca8-4f1a-bce7-59d2cfa1a24f None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.105764 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058323e-6ca8-4f1a-bce7-59d2cfa1a24f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.106035 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058323e-6ca8-4f1a-bce7-59d2cfa1a24f None admin] PATH_INFO: `/v3/endpoints` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.106774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058323e-6ca8-4f1a-bce7-59d2cfa1a24f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:41.108422 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058323e-6ca8-4f1a-bce7-59d2cfa1a24f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:41.126927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 527/1018] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:16:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:16:41.128218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709f2f-52c1-4dcc-88de-353194e318e4 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.128510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709f2f-52c1-4dcc-88de-353194e318e4 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.128784 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf709f2f-52c1-4dcc-88de-353194e318e4 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.130132 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 528/1019] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:41.143356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c9cd-2fe3-46b5-9d0c-6784d04e03ef None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.143739 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c9cd-2fe3-46b5-9d0c-6784d04e03ef None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.144022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c9cd-2fe3-46b5-9d0c-6784d04e03ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.144267 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4964232-887b-4be7-9059-30a8dd3ac4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:41.145115 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4964232-887b-4be7-9059-30a8dd3ac4bf None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.145473 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4964232-887b-4be7-9059-30a8dd3ac4bf None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.145676 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4964232-887b-4be7-9059-30a8dd3ac4bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.146965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4964232-887b-4be7-9059-30a8dd3ac4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:41.148942 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4964232-887b-4be7-9059-30a8dd3ac4bf None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:41.155825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 492/1020] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:41.156840 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4964232-887b-4be7-9059-30a8dd3ac4bf None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:41.164030 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-6420c9cd-2fe3-46b5-9d0c-6784d04e03ef None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:41.165478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-6420c9cd-2fe3-46b5-9d0c-6784d04e03ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:41.240012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 529/1021] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:41.263017 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c9cd-2fe3-46b5-9d0c-6784d04e03ef None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:41.267229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4964232-887b-4be7-9059-30a8dd3ac4bf None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:41.303419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472625-86c4-48f4-bd6c-cce8f363c35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:41.304144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472625-86c4-48f4-bd6c-cce8f363c35d None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.304433 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472625-86c4-48f4-bd6c-cce8f363c35d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.304702 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472625-86c4-48f4-bd6c-cce8f363c35d None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.305442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78472625-86c4-48f4-bd6c-cce8f363c35d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:41.307304 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78472625-86c4-48f4-bd6c-cce8f363c35d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:41.334991 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 493/1022] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:16:41 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:41.378363 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c9cd-2fe3-46b5-9d0c-6784d04e03ef None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:41.416736 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:41.417711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.417997 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.418271 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] PATH_INFO: `/v3/services/image` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.418950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:41.420358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:41.434826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:41.435776 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:41.437073 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 530/1023] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:16:41 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:41.444802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472625-86c4-48f4-bd6c-cce8f363c35d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:41.553874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472625-86c4-48f4-bd6c-cce8f363c35d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:41.587497 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:41.588097 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.588375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.588648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.589386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] RBAC: Adding query filter params (name=image) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:41.589690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:41.591056 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:41.596309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 494/1024] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:41.602319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:41.697261 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9efafa-04da-4442-ba74-f7fc51aaa3e0 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:41.732602 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:41.733289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.733571 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.733843 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.734551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] RBAC: Adding query filter params (type=image) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:41.734854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:41.736207 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:41.741651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 531/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:41.747824 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:41.844060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acbf5d-6a78-40c3-8e1d-bb42cf6bbe60 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:41.876941 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a3c9f-9dc7-4e4a-b3cb-fa56647fbc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:41.877622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a3c9f-9dc7-4e4a-b3cb-fa56647fbc44 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:41.877901 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a3c9f-9dc7-4e4a-b3cb-fa56647fbc44 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:41.878168 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a3c9f-9dc7-4e4a-b3cb-fa56647fbc44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:41.878946 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a3c9f-9dc7-4e4a-b3cb-fa56647fbc44 None admin] RBAC: Adding query filter params (interface=public, service_id=a84f5754fa0343528a41d6d58fd8562f, region_id=RegionOne) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:41.879260 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a3c9f-9dc7-4e4a-b3cb-fa56647fbc44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:41.880593 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a3c9f-9dc7-4e4a-b3cb-fa56647fbc44 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:41.890108 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 495/1026] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:16:41 2022] GET /identity/v3/endpoints?service_id=a84f5754fa0343528a41d6d58fd8562f&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:41.891120 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:41.987862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f505fac-2c9c-4519-853e-2f91d42d5b02 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:42.019813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f88e31-9f22-4a34-8cc9-958da1679d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:42.020471 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88e31-9f22-4a34-8cc9-958da1679d1e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:42.020745 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88e31-9f22-4a34-8cc9-958da1679d1e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:42.021016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f88e31-9f22-4a34-8cc9-958da1679d1e None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:42.021727 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f88e31-9f22-4a34-8cc9-958da1679d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:42.023603 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f88e31-9f22-4a34-8cc9-958da1679d1e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:42.037198 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 532/1027] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:16:41 2022] GET /identity/v3/services => generated 1957 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:16:43.485151 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7538-02b3-4d19-a859-8173fe18dcfd None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:43.485697 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7538-02b3-4d19-a859-8173fe18dcfd None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:43.485697 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a7538-02b3-4d19-a859-8173fe18dcfd None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:43.487199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 496/1028] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:43.492600 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699fbfb-c5a8-46cb-bec2-81b4d96cf900 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:43.493285 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699fbfb-c5a8-46cb-bec2-81b4d96cf900 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:43.493454 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699fbfb-c5a8-46cb-bec2-81b4d96cf900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:43.510062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-8699fbfb-c5a8-46cb-bec2-81b4d96cf900 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:43.511163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-8699fbfb-c5a8-46cb-bec2-81b4d96cf900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:43.578180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 533/1029] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:43.602434 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7538-02b3-4d19-a859-8173fe18dcfd None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:43.663461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00cce6-e725-4d73-bda3-871b1e132db5 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:43.663763 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00cce6-e725-4d73-bda3-871b1e132db5 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:43.664016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00cce6-e725-4d73-bda3-871b1e132db5 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:43.665448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 534/1030] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:43.671077 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e545ad4-e71b-4835-b74a-0f9f76a99928 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:43.671361 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e545ad4-e71b-4835-b74a-0f9f76a99928 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:43.671639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e545ad4-e71b-4835-b74a-0f9f76a99928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:43.688290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-0e545ad4-e71b-4835-b74a-0f9f76a99928 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:43.689526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-0e545ad4-e71b-4835-b74a-0f9f76a99928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:43.708106 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a7538-02b3-4d19-a859-8173fe18dcfd None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:43.751811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6d4da-ab89-462a-b922-1b22c4a3b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:43.752771 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6d4da-ab89-462a-b922-1b22c4a3b851 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:43.753175 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6d4da-ab89-462a-b922-1b22c4a3b851 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:43.753458 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6d4da-ab89-462a-b922-1b22c4a3b851 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:43.768021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 535/1031] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:43.774336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6d4da-ab89-462a-b922-1b22c4a3b851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-63f6d4da-ab89-462a-b922-1b22c4a3b851 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:43.785414 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:43.786310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:43.787054 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:43.788540 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 497/1032] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:16:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:43.794278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e545ad4-e71b-4835-b74a-0f9f76a99928 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:43.794432 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6d4da-ab89-462a-b922-1b22c4a3b851 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:43.899321 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e545ad4-e71b-4835-b74a-0f9f76a99928 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:43.899507 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6d4da-ab89-462a-b922-1b22c4a3b851 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:43.932381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332c139-8e3b-4c4d-97e6-39165970eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:43.933206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c139-8e3b-4c4d-97e6-39165970eb64 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:43.933495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c139-8e3b-4c4d-97e6-39165970eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:43.933769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c139-8e3b-4c4d-97e6-39165970eb64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:43.934932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332c139-8e3b-4c4d-97e6-39165970eb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:43.936357 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332c139-8e3b-4c4d-97e6-39165970eb64 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:43.937830 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:43.938495 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:43.938773 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:43.939043 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:43.939785 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:43.940413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:43.941774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:43.942811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 498/1033] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:43.943421 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332c139-8e3b-4c4d-97e6-39165970eb64 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:43.951492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 536/1034] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:16:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:43.956628 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.044049 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332c139-8e3b-4c4d-97e6-39165970eb64 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.055176 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a493b-dc98-46f6-bce7-46bd576e9ac2 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.076616 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8861fe3-b98b-489d-a918-547384f1431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.077323 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8861fe3-b98b-489d-a918-547384f1431d None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.077647 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8861fe3-b98b-489d-a918-547384f1431d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.077932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8861fe3-b98b-489d-a918-547384f1431d None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.078641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8861fe3-b98b-489d-a918-547384f1431d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.080081 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8861fe3-b98b-489d-a918-547384f1431d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.091812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d78b-1ee8-45eb-b38c-329721f37481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.092730 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d78b-1ee8-45eb-b38c-329721f37481 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.093144 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d78b-1ee8-45eb-b38c-329721f37481 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.093430 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d78b-1ee8-45eb-b38c-329721f37481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.113070 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 499/1035] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:16:43 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:44.113292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d78b-1ee8-45eb-b38c-329721f37481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.113292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d78b-1ee8-45eb-b38c-329721f37481 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-49c8d78b-1ee8-45eb-b38c-329721f37481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:44.128742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:44.129741 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:44.130566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:44.130566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:44.130566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:44.130566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:44.133413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 537/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:44.138305 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8861fe3-b98b-489d-a918-547384f1431d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.262043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8861fe3-b98b-489d-a918-547384f1431d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.296703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.297332 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.297612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.297883 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.298611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:44.298915 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.300287 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.315134 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 500/1037] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:44.316202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cae5f4-6867-49f4-8e4b-001fc32a1afc None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.316497 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cae5f4-6867-49f4-8e4b-001fc32a1afc None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.316781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cae5f4-6867-49f4-8e4b-001fc32a1afc None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.318250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 538/1038] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:44.320974 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.324051 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb6099-48e9-4db2-bb8c-db9a19e12ba6 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.324334 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb6099-48e9-4db2-bb8c-db9a19e12ba6 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.324607 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb6099-48e9-4db2-bb8c-db9a19e12ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.342122 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-00eb6099-48e9-4db2-bb8c-db9a19e12ba6 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:44.343296 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-00eb6099-48e9-4db2-bb8c-db9a19e12ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:44.415051 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 539/1039] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:44.430127 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee14826-9659-4cab-87a7-7939451323fe None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.438229 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb6099-48e9-4db2-bb8c-db9a19e12ba6 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.466383 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.467202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.467487 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.467765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.476426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.478334 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:44.496322 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:44.497862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:44.498888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 501/1040] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:16:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:44.505224 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.557510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb6099-48e9-4db2-bb8c-db9a19e12ba6 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.591940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.592738 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.593017 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.593318 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] PATH_INFO: `/v3/services/image` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.593972 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.595343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.602575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0e47b-f5b7-4299-aafe-824b92936eaf None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:44.607146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:44.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:44.614088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 540/1041] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:16:44 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:44.614783 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.637017 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.637716 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.638000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.638270 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.639099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:44.639402 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.640817 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.663927 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 502/1042] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:16:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:44.665363 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.718898 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b4715-1c86-48f1-9713-b64d1ca9bb04 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.752092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.752754 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.753047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.753336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.754048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] RBAC: Adding query filter params (name=image) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:44.754354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.755709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.760457 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 541/1043] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:44.766005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.781099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27880a-e26e-44e7-b570-c27bf0bd63a0 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.817864 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147a726-9aef-4eec-b8fc-f5dfc5d6047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.818437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a726-9aef-4eec-b8fc-f5dfc5d6047f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.818742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a726-9aef-4eec-b8fc-f5dfc5d6047f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.819012 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a726-9aef-4eec-b8fc-f5dfc5d6047f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.864789 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf9e1c-cfcb-44c9-9a42-c8a686e1bf7c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:44.884696 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147a726-9aef-4eec-b8fc-f5dfc5d6047f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.886149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147a726-9aef-4eec-b8fc-f5dfc5d6047f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.898678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:44.899346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:44.899630 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:44.899912 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:44.900631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] RBAC: Adding query filter params (type=image) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:44.900929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:44.902337 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:44.907083 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 542/1044] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:44.913323 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:44.988954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 503/1045] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:16:44.994859 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147a726-9aef-4eec-b8fc-f5dfc5d6047f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:45.013152 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8820a1b-290a-4652-bd5f-5ff38e986b72 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:45.048259 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a837b4-cf22-4502-8d3a-e100811fd854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:45.048928 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a837b4-cf22-4502-8d3a-e100811fd854 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:45.049241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a837b4-cf22-4502-8d3a-e100811fd854 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:45.049518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a837b4-cf22-4502-8d3a-e100811fd854 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:45.050262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a837b4-cf22-4502-8d3a-e100811fd854 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:45.051635 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a837b4-cf22-4502-8d3a-e100811fd854 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:45.074201 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 543/1046] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:16:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:16:45.101297 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147a726-9aef-4eec-b8fc-f5dfc5d6047f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:45.145963 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:45.147386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:45.147672 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:45.147947 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:45.159543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:45.161030 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:45.180880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:45.182000 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:45.182796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:45.182796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:45.182796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:45.182796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:45.183598 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 504/1047] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:45.188537 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a837b4-cf22-4502-8d3a-e100811fd854 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:45.301483 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a837b4-cf22-4502-8d3a-e100811fd854 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:45.335559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:45.336203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:45.336489 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:45.336767 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:45.337541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:45.337853 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:45.339241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:45.351559 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 544/1048] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:45.357158 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:45.455082 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a03ee-8a9f-4784-8d8a-ff7baf8fec04 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:45.488331 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:45.489173 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:45.489457 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:45.489725 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:45.496706 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:45.498170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:45.512128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:45.513230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:45.514016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 505/1049] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:45.522838 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:45.620884 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbc2db-809f-4a54-b7c4-b3ae23c6c2fb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:45.654924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4da341-cc03-495a-87e4-bcbf8715b4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:45.655583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da341-cc03-495a-87e4-bcbf8715b4db None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:45.655863 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da341-cc03-495a-87e4-bcbf8715b4db None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:45.656138 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da341-cc03-495a-87e4-bcbf8715b4db None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:45.656918 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da341-cc03-495a-87e4-bcbf8715b4db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:45.657255 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da341-cc03-495a-87e4-bcbf8715b4db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:45.658679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da341-cc03-495a-87e4-bcbf8715b4db None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:45.675021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 545/1050] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:45.676620 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:45.777683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3b46b-10cf-4458-9966-350320ba6615 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:45.811881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c9a4a-c285-43e0-8b53-80ccd5ace775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:45.812535 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c9a4a-c285-43e0-8b53-80ccd5ace775 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:45.812814 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c9a4a-c285-43e0-8b53-80ccd5ace775 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:45.813111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c9a4a-c285-43e0-8b53-80ccd5ace775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:45.813936 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c9a4a-c285-43e0-8b53-80ccd5ace775 None admin] RBAC: Adding query filter params (role.id=5bdec21d519c4a929fc1ddde15a7b9ca, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=f3a73bc43c23400d8ee7d4e880aa7242) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:45.814246 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c9a4a-c285-43e0-8b53-80ccd5ace775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:45.815674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c9a4a-c285-43e0-8b53-80ccd5ace775 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:45.822898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 506/1051] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:45 2022] GET /identity/v3/role_assignments?user.id=f3a73bc43c23400d8ee7d4e880aa7242&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=5bdec21d519c4a929fc1ddde15a7b9ca => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:46.465404 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d2d6-0d32-4ef4-b462-fd0b863434d5 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:46.465708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d2d6-0d32-4ef4-b462-fd0b863434d5 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:46.465964 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d2d6-0d32-4ef4-b462-fd0b863434d5 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:46.467492 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 546/1052] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:46.472504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708cf0-d5f8-4032-bb12-09a67abaa00f None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:46.472866 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708cf0-d5f8-4032-bb12-09a67abaa00f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:46.473255 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708cf0-d5f8-4032-bb12-09a67abaa00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:46.493865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-07708cf0-d5f8-4032-bb12-09a67abaa00f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:46.495048 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-07708cf0-d5f8-4032-bb12-09a67abaa00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:46.561152 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 507/1053] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:46.581722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d2d6-0d32-4ef4-b462-fd0b863434d5 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:46.680772 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d2d6-0d32-4ef4-b462-fd0b863434d5 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:46.712985 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591e407-aaa7-446d-aed8-9f30eb485fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:46.713801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591e407-aaa7-446d-aed8-9f30eb485fe7 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:46.714084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591e407-aaa7-446d-aed8-9f30eb485fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:46.714359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591e407-aaa7-446d-aed8-9f30eb485fe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:46.724540 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591e407-aaa7-446d-aed8-9f30eb485fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-5591e407-aaa7-446d-aed8-9f30eb485fe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:46.732651 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:46.733531 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:46.739654 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 547/1054] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:46.740296 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07708cf0-d5f8-4032-bb12-09a67abaa00f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:46.835992 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07708cf0-d5f8-4032-bb12-09a67abaa00f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:46.868957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:46.869653 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:46.869935 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:46.870203 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:46.870952 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:46.871254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:46.872615 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:46.885966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 508/1055] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:46.887446 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591e407-aaa7-446d-aed8-9f30eb485fe7 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:46.988453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591e407-aaa7-446d-aed8-9f30eb485fe7 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:47.021396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e81872-b09f-4a10-b3f9-f594f929f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:47.022848 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e81872-b09f-4a10-b3f9-f594f929f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:47.023123 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e81872-b09f-4a10-b3f9-f594f929f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:47.023396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e81872-b09f-4a10-b3f9-f594f929f9f4 None admin] PATH_INFO: `/v3/users/b61ff532726d44849f98a5e683f29b18` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:47.036528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e81872-b09f-4a10-b3f9-f594f929f9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=b61ff532726d44849f98a5e683f29b18)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:47.038203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e81872-b09f-4a10-b3f9-f594f929f9f4 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:47.044622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 548/1056] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:16:46 2022] GET /identity/v3/users/b61ff532726d44849f98a5e683f29b18 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:47.045703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:47.143500 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b7fdb-6e32-46d3-b587-3d435f94392d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:47.179095 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ba272-c902-4d5c-bce2-40391eab4e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:47.179748 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ba272-c902-4d5c-bce2-40391eab4e7a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:47.180028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ba272-c902-4d5c-bce2-40391eab4e7a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:47.180302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ba272-c902-4d5c-bce2-40391eab4e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:47.245476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ba272-c902-4d5c-bce2-40391eab4e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:47.246923 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ba272-c902-4d5c-bce2-40391eab4e7a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:47.323900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24b023-5d0c-4a74-a6df-910f57d2e497 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:47.324239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24b023-5d0c-4a74-a6df-910f57d2e497 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:47.324455 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24b023-5d0c-4a74-a6df-910f57d2e497 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:47.326093 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 549/1057] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:47.330817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a3818-9ddd-42e4-ab0f-20250ec754a6 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:47.331099 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a3818-9ddd-42e4-ab0f-20250ec754a6 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:47.331378 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a3818-9ddd-42e4-ab0f-20250ec754a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:47.348751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 509/1058] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:16:47.349891 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ba272-c902-4d5c-bce2-40391eab4e7a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:47.350608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-9d1a3818-9ddd-42e4-ab0f-20250ec754a6 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:47.351802 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-9d1a3818-9ddd-42e4-ab0f-20250ec754a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:47.417071 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 550/1059] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:47.439209 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a3818-9ddd-42e4-ab0f-20250ec754a6 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:47.443581 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ba272-c902-4d5c-bce2-40391eab4e7a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:47.477476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3547b-4110-422c-b957-c61feb21073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:47.478330 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3547b-4110-422c-b957-c61feb21073e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:47.478612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3547b-4110-422c-b957-c61feb21073e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:47.478884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3547b-4110-422c-b957-c61feb21073e None admin] PATH_INFO: `/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:47.480070 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3547b-4110-422c-b957-c61feb21073e None admin] RBAC: Authorizing `identity:get_project(project_id=b9304b5154504dd6931ba78f3cf3ca5b)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:47.481567 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3547b-4110-422c-b957-c61feb21073e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:47.483447 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 510/1060] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:16:47 2022] GET /identity/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:47.489209 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3547b-4110-422c-b957-c61feb21073e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:47.547639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a3818-9ddd-42e4-ab0f-20250ec754a6 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:47.583509 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b687635-f194-45f8-86c3-2e3e9e317927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:47.584184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687635-f194-45f8-86c3-2e3e9e317927 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:47.584461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687635-f194-45f8-86c3-2e3e9e317927 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:47.584729 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687635-f194-45f8-86c3-2e3e9e317927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:47.597790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3547b-4110-422c-b957-c61feb21073e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:47.633099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:47.633757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:47.634034 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:47.634303 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:47.635128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=b9304b5154504dd6931ba78f3cf3ca5b, user.id=b61ff532726d44849f98a5e683f29b18) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:47.635431 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:47.636930 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:47.653353 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 511/1061] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:47 2022] GET /identity/v3/role_assignments?user.id=b61ff532726d44849f98a5e683f29b18&scope.project.id=b9304b5154504dd6931ba78f3cf3ca5b&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:47.653579 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b687635-f194-45f8-86c3-2e3e9e317927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:47.653579 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b687635-f194-45f8-86c3-2e3e9e317927 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:47.755616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 551/1062] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:16:47.761701 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:47.878504 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522de7f4-dce1-494f-9726-20eecef9172e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:47.923002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40555a5-da71-4bc5-ba96-8d72452e2dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:47.923800 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40555a5-da71-4bc5-ba96-8d72452e2dc1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:47.924085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40555a5-da71-4bc5-ba96-8d72452e2dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:47.924360 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40555a5-da71-4bc5-ba96-8d72452e2dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:47.935742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40555a5-da71-4bc5-ba96-8d72452e2dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:47.937214 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40555a5-da71-4bc5-ba96-8d72452e2dc1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-c40555a5-da71-4bc5-ba96-8d72452e2dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:47.953846 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:47.954657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:47.955475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:47.955475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:47.955475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:47.955475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:47.955475 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 512/1063] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:47.962100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b687635-f194-45f8-86c3-2e3e9e317927 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:48.060623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b687635-f194-45f8-86c3-2e3e9e317927 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:48.075336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3296e2-8342-4f0d-ad43-a39701403852 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:48.075631 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3296e2-8342-4f0d-ad43-a39701403852 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:48.075916 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3296e2-8342-4f0d-ad43-a39701403852 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:48.077468 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 513/1064] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:48.082383 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f21ca-d053-443d-b5e6-7b0fe920986b None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:48.082673 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f21ca-d053-443d-b5e6-7b0fe920986b None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:48.082957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f21ca-d053-443d-b5e6-7b0fe920986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:48.093494 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:48.094167 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:48.094448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:48.094730 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:48.095449 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:48.095757 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:48.097141 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:48.100678 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-6b6f21ca-d053-443d-b5e6-7b0fe920986b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:48.101906 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-6b6f21ca-d053-443d-b5e6-7b0fe920986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:48.117070 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 552/1065] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:48.184892 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 514/1066] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:48.212667 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:48.330623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94eaff1-54ca-4f9c-b245-5b86f8d4b923 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:48.364037 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889e89-8f43-45b3-906e-10f403722857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:48.364854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889e89-8f43-45b3-906e-10f403722857 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:48.365162 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889e89-8f43-45b3-906e-10f403722857 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:48.365469 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a889e89-8f43-45b3-906e-10f403722857 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:48.376029 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a889e89-8f43-45b3-906e-10f403722857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-2a889e89-8f43-45b3-906e-10f403722857 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:48.384926 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:48.386021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:48.386921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 553/1067] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:16:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:48.392733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f21ca-d053-443d-b5e6-7b0fe920986b None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:48.493283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f21ca-d053-443d-b5e6-7b0fe920986b None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:48.525765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:48.526425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:48.526702 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:48.526969 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:48.527705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:48.528007 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:48.529389 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:48.537589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 515/1068] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:16:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:48.544337 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889e89-8f43-45b3-906e-10f403722857 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:48.655950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a889e89-8f43-45b3-906e-10f403722857 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:48.687234 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:48.688022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:48.688315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:48.688566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:48.699283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:48.700697 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:48.716927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:48.717892 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:48.718707 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:48.718707 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:48.718707 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:48.718707 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:48.718707 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 554/1069] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:48.723738 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:48.821548 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8ace4-c6f2-4a3d-b71b-35df7cce0026 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:48.852459 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:48.853139 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:48.853419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:48.853689 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:48.854397 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:48.854696 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:48.856039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:48.868869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 516/1070] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:48.874555 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:48.968966 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df6006-9902-4e16-9480-620bb5d8a04c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:49.000232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:49.001010 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.001319 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.001591 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.009217 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:49.010633 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:49.024491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:49.025516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:49.026608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:49.029203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 555/1071] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:16:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:49.031476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:49.126389 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e235e-de8c-402d-87be-0ea3183335ed None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:49.157836 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:49.158480 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.158757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.159025 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.159841 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:49.160142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:49.161686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:49.179024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 517/1072] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:16:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:49.184551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:49.283305 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4f64e-3c5c-499f-b559-95d764047d43 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:49.315173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:49.315949 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.316227 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.316498 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.326634 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:49.328056 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:49.344263 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:49.345249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:49.346262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:49.346262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:49.346262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:49.346262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:49.349196 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 556/1073] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:49.351701 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:49.446408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce151c7-bdde-4d87-9908-20fa3b2653f0 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:49.479020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:49.479674 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.479951 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.480221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.480933 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:49.481341 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:49.482689 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:49.494511 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 518/1074] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:49.500156 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:49.595375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee15e6-9a19-437c-a845-94f7b795570b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:49.627314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:49.628088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.628367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.628638 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.635557 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:49.636977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:49.650629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:49.651528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:49.652339 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 557/1075] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:49.657652 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:49.757195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5e43a-01a7-407b-ac3f-a95b5bc211d2 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:49.789380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622dff9-2cd8-4df0-a8b9-5fe461add8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:49.790032 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622dff9-2cd8-4df0-a8b9-5fe461add8c8 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.790310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622dff9-2cd8-4df0-a8b9-5fe461add8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.790582 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622dff9-2cd8-4df0-a8b9-5fe461add8c8 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.791356 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622dff9-2cd8-4df0-a8b9-5fe461add8c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:49.791661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622dff9-2cd8-4df0-a8b9-5fe461add8c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:49.793082 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622dff9-2cd8-4df0-a8b9-5fe461add8c8 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:49.804490 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 519/1076] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:49.810490 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:49.885543 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c365d71-30a2-4991-aeb6-96cf8fd0870f None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.885759 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c365d71-30a2-4991-aeb6-96cf8fd0870f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.886038 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c365d71-30a2-4991-aeb6-96cf8fd0870f None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.887405 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 520/1077] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:49.892396 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6be2a2-78aa-4244-b613-30e588c49ea8 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.892681 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6be2a2-78aa-4244-b613-30e588c49ea8 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.892988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6be2a2-78aa-4244-b613-30e588c49ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.905933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400897d6-528e-4df1-a247-a3706636db45 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:49.908831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-4a6be2a2-78aa-4244-b613-30e588c49ea8 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:49.910034 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-4a6be2a2-78aa-4244-b613-30e588c49ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:49.939814 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52388c1-0dd6-4757-bbb7-900f004ddf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:49.940780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52388c1-0dd6-4757-bbb7-900f004ddf5a None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:49.941081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52388c1-0dd6-4757-bbb7-900f004ddf5a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:49.941357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52388c1-0dd6-4757-bbb7-900f004ddf5a None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/f3a73bc43c23400d8ee7d4e880aa7242/roles/5bdec21d519c4a929fc1ddde15a7b9ca` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:49.944012 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52388c1-0dd6-4757-bbb7-900f004ddf5a None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=f3a73bc43c23400d8ee7d4e880aa7242, role_id=5bdec21d519c4a929fc1ddde15a7b9ca)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:49.945459 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52388c1-0dd6-4757-bbb7-900f004ddf5a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:49.957753 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 558/1078] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:16:49 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/f3a73bc43c23400d8ee7d4e880aa7242/roles/5bdec21d519c4a929fc1ddde15a7b9ca => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:16:49.978406 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 521/1079] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:50.002244 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52388c1-0dd6-4757-bbb7-900f004ddf5a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:50.106406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52388c1-0dd6-4757-bbb7-900f004ddf5a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:50.143177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2d918-a91a-43d4-a70a-144dcd767864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:50.143903 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2d918-a91a-43d4-a70a-144dcd767864 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:50.144188 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2d918-a91a-43d4-a70a-144dcd767864 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:50.144459 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2d918-a91a-43d4-a70a-144dcd767864 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:50.155182 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2d918-a91a-43d4-a70a-144dcd767864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-69c2d918-a91a-43d4-a70a-144dcd767864 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:50.162942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:50.163936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:50.164708 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 559/1080] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:50.170324 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6be2a2-78aa-4244-b613-30e588c49ea8 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:50.269085 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6be2a2-78aa-4244-b613-30e588c49ea8 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:50.301162 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:50.301827 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:50.302109 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:50.302379 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:50.303637 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:50.303938 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:50.305311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:50.313129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 522/1081] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:50.319574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2d918-a91a-43d4-a70a-144dcd767864 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:50.416867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2d918-a91a-43d4-a70a-144dcd767864 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:50.449518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3f09f-2033-42f9-a4c3-ad941f86c83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:50.450381 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3f09f-2033-42f9-a4c3-ad941f86c83b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:50.450663 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3f09f-2033-42f9-a4c3-ad941f86c83b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:50.450936 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3f09f-2033-42f9-a4c3-ad941f86c83b None admin] PATH_INFO: `/v3/users/b61ff532726d44849f98a5e683f29b18` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:50.452127 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3f09f-2033-42f9-a4c3-ad941f86c83b None admin] RBAC: Authorizing `identity:get_user(user_id=b61ff532726d44849f98a5e683f29b18)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:50.453581 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3f09f-2033-42f9-a4c3-ad941f86c83b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:50.455221 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 560/1082] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:16:50 2022] GET /identity/v3/users/b61ff532726d44849f98a5e683f29b18 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:50.461080 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:50.556262 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34d02-cca5-41f0-a103-499a5b4eac6c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:50.587888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d87fcc-b7c6-4a2c-90e9-1f203017c024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:50.588769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87fcc-b7c6-4a2c-90e9-1f203017c024 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:50.589073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87fcc-b7c6-4a2c-90e9-1f203017c024 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:50.589347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87fcc-b7c6-4a2c-90e9-1f203017c024 None admin] PATH_INFO: `/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:50.590520 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d87fcc-b7c6-4a2c-90e9-1f203017c024 None admin] RBAC: Authorizing `identity:get_project(project_id=b9304b5154504dd6931ba78f3cf3ca5b)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:50.591939 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d87fcc-b7c6-4a2c-90e9-1f203017c024 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:50.593651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 523/1083] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:16:50 2022] GET /identity/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:50.599484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3f09f-2033-42f9-a4c3-ad941f86c83b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:50.693890 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3f09f-2033-42f9-a4c3-ad941f86c83b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:50.725514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e37b78-9940-4007-852b-0d7bff0a0862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:50.726393 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e37b78-9940-4007-852b-0d7bff0a0862 None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:50.726673 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e37b78-9940-4007-852b-0d7bff0a0862 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:50.726947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e37b78-9940-4007-852b-0d7bff0a0862 None admin] PATH_INFO: `/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b/users/b61ff532726d44849f98a5e683f29b18/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:50.728890 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e37b78-9940-4007-852b-0d7bff0a0862 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9304b5154504dd6931ba78f3cf3ca5b, user_id=b61ff532726d44849f98a5e683f29b18, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:50.730352 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e37b78-9940-4007-852b-0d7bff0a0862 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:50.743348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 561/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:16:50 2022] PUT /identity/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b/users/b61ff532726d44849f98a5e683f29b18/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:16:52.158682 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e28bd-0060-48e1-a32e-a53b637fdd72 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:52.159244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e28bd-0060-48e1-a32e-a53b637fdd72 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:52.159244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e28bd-0060-48e1-a32e-a53b637fdd72 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:52.160657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 524/1085] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:52.166510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23cdaf-67dd-449a-b1de-5ce3cc93b00f None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:52.166920 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23cdaf-67dd-449a-b1de-5ce3cc93b00f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:52.167064 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23cdaf-67dd-449a-b1de-5ce3cc93b00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:52.183670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-db23cdaf-67dd-449a-b1de-5ce3cc93b00f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:52.184828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-db23cdaf-67dd-449a-b1de-5ce3cc93b00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:52.247517 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 562/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:52.270263 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e28bd-0060-48e1-a32e-a53b637fdd72 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:52.366535 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e28bd-0060-48e1-a32e-a53b637fdd72 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:52.398025 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115330e-94fa-47a6-be9d-cdfb84ae18ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:52.398823 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115330e-94fa-47a6-be9d-cdfb84ae18ca None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:52.399097 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115330e-94fa-47a6-be9d-cdfb84ae18ca None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:52.399365 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115330e-94fa-47a6-be9d-cdfb84ae18ca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:52.409103 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115330e-94fa-47a6-be9d-cdfb84ae18ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-7115330e-94fa-47a6-be9d-cdfb84ae18ca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:52.417741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:52.418607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:52.419415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 525/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:16:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:52.424847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23cdaf-67dd-449a-b1de-5ce3cc93b00f None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:52.522576 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23cdaf-67dd-449a-b1de-5ce3cc93b00f None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:52.554719 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:52.555380 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:52.555657 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:52.555927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:52.556668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:52.556969 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:52.558367 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:52.566040 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 563/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:16:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:52.571690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115330e-94fa-47a6-be9d-cdfb84ae18ca None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:52.666921 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115330e-94fa-47a6-be9d-cdfb84ae18ca None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:52.698291 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:52.699068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:52.699347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:52.699618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:52.710144 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:52.711569 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:52.728355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:52.729171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:52.730087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:52.730087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:52.730087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:52.730087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:52.730087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 526/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:52.735259 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:52.835466 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9002f-91fe-4b69-b9bb-e6a5102178c9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:52.866636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505b8f6-a8de-4e21-9424-05eaf4bcf535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:52.867289 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505b8f6-a8de-4e21-9424-05eaf4bcf535 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:52.867570 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505b8f6-a8de-4e21-9424-05eaf4bcf535 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:52.868114 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505b8f6-a8de-4e21-9424-05eaf4bcf535 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:52.868833 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505b8f6-a8de-4e21-9424-05eaf4bcf535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:52.869186 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505b8f6-a8de-4e21-9424-05eaf4bcf535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:52.870545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505b8f6-a8de-4e21-9424-05eaf4bcf535 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:52.882213 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 564/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:52.887922 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:52.944764 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de286142-b240-405a-9a31-fc640b35189f None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:52.944970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de286142-b240-405a-9a31-fc640b35189f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:52.945278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de286142-b240-405a-9a31-fc640b35189f None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:52.946663 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 565/1091] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:16:52.951586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744322-f0e2-4bda-866c-6abc8e977247 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:52.951869 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744322-f0e2-4bda-866c-6abc8e977247 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:52.952143 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04744322-f0e2-4bda-866c-6abc8e977247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:52.968583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-04744322-f0e2-4bda-866c-6abc8e977247 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:52.969801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-04744322-f0e2-4bda-866c-6abc8e977247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:52.985285 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494eca-9e0c-4682-adf6-aed4fa6b0fd6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.017406 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.018185 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.018461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.018729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.026454 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.027882 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.032940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 566/1092] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:16:53.041469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:53.042351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:53.043202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 527/1093] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:16:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:16:53.048254 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04744322-f0e2-4bda-866c-6abc8e977247 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.056110 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.141604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04744322-f0e2-4bda-866c-6abc8e977247 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.150374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104d60b-e7df-4b99-a4cd-f85d7f1b68f0 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.174041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.174707 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.174985 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.175258 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.176088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:53.176391 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.178398 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.182563 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7840682-d332-49c6-baed-c056c3566bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.183344 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840682-d332-49c6-baed-c056c3566bcf None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.183622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840682-d332-49c6-baed-c056c3566bcf None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.183892 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840682-d332-49c6-baed-c056c3566bcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.193605 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7840682-d332-49c6-baed-c056c3566bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:16:53.194644 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 567/1094] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:53.200311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-a7840682-d332-49c6-baed-c056c3566bcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:16:53.201178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:53.202036 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:16:53.202798 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 528/1095] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:53.207989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7840682-d332-49c6-baed-c056c3566bcf None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.308187 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7840682-d332-49c6-baed-c056c3566bcf None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.309443 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a63b9-6f41-4dda-9da4-ee5d0521022b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.340500 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.341196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.341482 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.341760 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.342233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cbc24-f2e9-41a3-84b3-6047249266c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.342507 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:53.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.342899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cbc24-f2e9-41a3-84b3-6047249266c8 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.343128 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cbc24-f2e9-41a3-84b3-6047249266c8 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.343396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cbc24-f2e9-41a3-84b3-6047249266c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.344176 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.352518 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 529/1096] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:53.358211 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.407300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cbc24-f2e9-41a3-84b3-6047249266c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.408734 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cbc24-f2e9-41a3-84b3-6047249266c8 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.451773 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca958b9-0c54-4247-be0c-f8c1c2a73e0d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.484014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4c7bb-f652-48fb-8876-050a6ce53d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.484874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c7bb-f652-48fb-8876-050a6ce53d0a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.485193 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c7bb-f652-48fb-8876-050a6ce53d0a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.485470 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c7bb-f652-48fb-8876-050a6ce53d0a None admin] PATH_INFO: `/v3/users/b61ff532726d44849f98a5e683f29b18` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.486666 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4c7bb-f652-48fb-8876-050a6ce53d0a None admin] RBAC: Authorizing `identity:get_user(user_id=b61ff532726d44849f98a5e683f29b18)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.488073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4c7bb-f652-48fb-8876-050a6ce53d0a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.489733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 530/1097] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/users/b61ff532726d44849f98a5e683f29b18 => generated 285 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:53.495620 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4c7bb-f652-48fb-8876-050a6ce53d0a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.501332 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 568/1098] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:16:53.507066 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cbc24-f2e9-41a3-84b3-6047249266c8 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.593317 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4c7bb-f652-48fb-8876-050a6ce53d0a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.605214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cbc24-f2e9-41a3-84b3-6047249266c8 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.627854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c23b-f1f5-4309-8ac1-29af4493c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.628510 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c23b-f1f5-4309-8ac1-29af4493c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.628794 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c23b-f1f5-4309-8ac1-29af4493c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.629095 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c23b-f1f5-4309-8ac1-29af4493c8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.640327 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.641151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.641431 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.641701 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.651960 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.653429 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:16:53.670238 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:53.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:53.672018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:16:53.672018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:16:53.672018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:16:53.672018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:53.672018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 569/1099] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:53.677177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.692638 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8c23b-f1f5-4309-8ac1-29af4493c8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.694612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8c23b-f1f5-4309-8ac1-29af4493c8b6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.769619 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153211cf-d981-4ffb-ad10-bd4b226c84b0 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.787451 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 531/1100] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:16:53.793236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c23b-f1f5-4309-8ac1-29af4493c8b6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.801606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.802262 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.802543 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.802850 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.803566 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:53.803873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.805253 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.815876 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 570/1101] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:53.821807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.885501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8c23b-f1f5-4309-8ac1-29af4493c8b6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.914802 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc788fb2-d3a3-438f-b513-4c6b781f91dd None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:53.918490 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b4b9a-8969-4d6a-8315-c7ea267e8ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.919461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b4b9a-8969-4d6a-8315-c7ea267e8ea2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.919792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b4b9a-8969-4d6a-8315-c7ea267e8ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.920236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b4b9a-8969-4d6a-8315-c7ea267e8ea2 None admin] PATH_INFO: `/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.921678 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b4b9a-8969-4d6a-8315-c7ea267e8ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=b9304b5154504dd6931ba78f3cf3ca5b)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.923102 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b4b9a-8969-4d6a-8315-c7ea267e8ea2 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.924959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 532/1102] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:53.931280 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b4b9a-8969-4d6a-8315-c7ea267e8ea2 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:53.947625 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:53.948411 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:53.948692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:53.948962 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:53.955362 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:53.956774 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:16:53.970486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:16:53.971477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:16:53.972508 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 571/1103] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:16:53.977397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:54.030907 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b4b9a-8969-4d6a-8315-c7ea267e8ea2 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:54.063444 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:54.064092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:54.064368 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:54.064638 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:54.065489 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=b9304b5154504dd6931ba78f3cf3ca5b, user.id=b61ff532726d44849f98a5e683f29b18) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:54.065802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:54.067209 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:54.070740 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d803a0-3e90-498d-b2ab-a93c97224818 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:54.073526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 533/1104] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/role_assignments?user.id=b61ff532726d44849f98a5e683f29b18&scope.project.id=b9304b5154504dd6931ba78f3cf3ca5b&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:16:54.105606 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55a60e-b739-4c1a-9a8d-65349b252001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:54.106284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55a60e-b739-4c1a-9a8d-65349b252001 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:54.106564 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55a60e-b739-4c1a-9a8d-65349b252001 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:54.106837 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55a60e-b739-4c1a-9a8d-65349b252001 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:54.107639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55a60e-b739-4c1a-9a8d-65349b252001 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:54.107947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55a60e-b739-4c1a-9a8d-65349b252001 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:54.109436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55a60e-b739-4c1a-9a8d-65349b252001 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:54.122665 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 572/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:16:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:54.128921 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:54.230912 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb57cdf-ea20-4d75-89b5-06a4b6ec5e3c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:54.266055 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a8fe8-b11e-4392-9db7-2ce65b1cb34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:54.266721 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a8fe8-b11e-4392-9db7-2ce65b1cb34c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:54.267004 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a8fe8-b11e-4392-9db7-2ce65b1cb34c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:54.267279 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a8fe8-b11e-4392-9db7-2ce65b1cb34c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:54.268118 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a8fe8-b11e-4392-9db7-2ce65b1cb34c None admin] RBAC: Adding query filter params (role.id=5bdec21d519c4a929fc1ddde15a7b9ca, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=f3a73bc43c23400d8ee7d4e880aa7242) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:54.268425 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a8fe8-b11e-4392-9db7-2ce65b1cb34c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:54.269887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a8fe8-b11e-4392-9db7-2ce65b1cb34c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:54.276235 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 534/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:54 2022] GET /identity/v3/role_assignments?user.id=f3a73bc43c23400d8ee7d4e880aa7242&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=5bdec21d519c4a929fc1ddde15a7b9ca => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:56.329624 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a9671-39e6-4c53-8a2b-07f0c6853586 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:56.330129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a9671-39e6-4c53-8a2b-07f0c6853586 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:56.330225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a9671-39e6-4c53-8a2b-07f0c6853586 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:56.331541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 573/1107] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:56.337302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba528171-dadc-42e3-8371-43e86cea7b6d None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:56.337692 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba528171-dadc-42e3-8371-43e86cea7b6d None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:56.337975 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba528171-dadc-42e3-8371-43e86cea7b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:56.354963 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-ba528171-dadc-42e3-8371-43e86cea7b6d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:56.356160 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-ba528171-dadc-42e3-8371-43e86cea7b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:56.419391 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 535/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:56.443610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a9671-39e6-4c53-8a2b-07f0c6853586 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:56.539707 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a9671-39e6-4c53-8a2b-07f0c6853586 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:56.571268 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf89f1-2712-4c74-9356-a82f5e582dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:56.572036 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf89f1-2712-4c74-9356-a82f5e582dab None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:56.572314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf89f1-2712-4c74-9356-a82f5e582dab None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:56.572585 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf89f1-2712-4c74-9356-a82f5e582dab None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:56.573746 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf89f1-2712-4c74-9356-a82f5e582dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:56.575157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf89f1-2712-4c74-9356-a82f5e582dab None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:56.576722 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 574/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:16:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:56.582239 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba528171-dadc-42e3-8371-43e86cea7b6d None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:56.675023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba528171-dadc-42e3-8371-43e86cea7b6d None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:56.706732 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66ee65-d402-421c-a222-bc4737ca78cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:56.707383 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66ee65-d402-421c-a222-bc4737ca78cb None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:56.707663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66ee65-d402-421c-a222-bc4737ca78cb None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:56.707937 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66ee65-d402-421c-a222-bc4737ca78cb None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:56.708650 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66ee65-d402-421c-a222-bc4737ca78cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:56.710645 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66ee65-d402-421c-a222-bc4737ca78cb None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:56.728654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 536/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:16:56 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:16:58.967164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2ca2-84e7-4828-92c5-55e87aee1d6f None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:58.967703 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2ca2-84e7-4828-92c5-55e87aee1d6f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:58.967703 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2ca2-84e7-4828-92c5-55e87aee1d6f None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:58.969101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 575/1111] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:16:58.974881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e5938-f65f-45d6-b0cf-ecee686f9a31 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:58.975258 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e5938-f65f-45d6-b0cf-ecee686f9a31 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:58.975493 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e5938-f65f-45d6-b0cf-ecee686f9a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:58.992321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-a83e5938-f65f-45d6-b0cf-ecee686f9a31 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:16:58.993805 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-a83e5938-f65f-45d6-b0cf-ecee686f9a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:16:59.057515 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 537/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:16:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:16:59.080131 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe2ca2-84e7-4828-92c5-55e87aee1d6f None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:59.175585 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe2ca2-84e7-4828-92c5-55e87aee1d6f None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:59.206792 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bf7b0-7f46-4d09-9f2b-598d0bef84de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:59.207623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf7b0-7f46-4d09-9f2b-598d0bef84de None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:59.207900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf7b0-7f46-4d09-9f2b-598d0bef84de None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:59.208170 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bf7b0-7f46-4d09-9f2b-598d0bef84de None admin] PATH_INFO: `/v3/roles/c8ee1507247e4b34a213a8033c46378f` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:59.210162 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bf7b0-7f46-4d09-9f2b-598d0bef84de None admin] RBAC: Authorizing `identity:get_role(role_id=c8ee1507247e4b34a213a8033c46378f)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:59.211516 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bf7b0-7f46-4d09-9f2b-598d0bef84de None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:59.212847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 576/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:16:59 2022] GET /identity/v3/roles/c8ee1507247e4b34a213a8033c46378f => generated 223 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:59.218447 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e5938-f65f-45d6-b0cf-ecee686f9a31 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:59.311435 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e5938-f65f-45d6-b0cf-ecee686f9a31 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:59.343215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592bfac-f43b-4118-8642-84edad1b8994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:59.344079 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592bfac-f43b-4118-8642-84edad1b8994 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:59.344363 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592bfac-f43b-4118-8642-84edad1b8994 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:59.344639 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592bfac-f43b-4118-8642-84edad1b8994 None admin] PATH_INFO: `/v3/users/1fab6c73afd74b7faa1282da7b357659` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:59.357607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592bfac-f43b-4118-8642-84edad1b8994 None admin] RBAC: Authorizing `identity:get_user(user_id=1fab6c73afd74b7faa1282da7b357659)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:59.359030 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592bfac-f43b-4118-8642-84edad1b8994 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:59.360663 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 538/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:16:59 2022] GET /identity/v3/users/1fab6c73afd74b7faa1282da7b357659 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:59.366966 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bf7b0-7f46-4d09-9f2b-598d0bef84de None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:59.460140 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bf7b0-7f46-4d09-9f2b-598d0bef84de None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:59.491347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a26e2-1c5a-484c-92cb-6b4467804bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:59.491924 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a26e2-1c5a-484c-92cb-6b4467804bed None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:59.492205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a26e2-1c5a-484c-92cb-6b4467804bed None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:59.492478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a26e2-1c5a-484c-92cb-6b4467804bed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:59.560736 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a26e2-1c5a-484c-92cb-6b4467804bed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:59.562202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a26e2-1c5a-484c-92cb-6b4467804bed None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:59.654164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 577/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:16:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:16:59.661104 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592bfac-f43b-4118-8642-84edad1b8994 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:59.754397 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592bfac-f43b-4118-8642-84edad1b8994 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:59.786833 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f8c87-340b-4860-8af4-f6160941d465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:59.787685 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f8c87-340b-4860-8af4-f6160941d465 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:59.787967 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f8c87-340b-4860-8af4-f6160941d465 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:59.788242 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f8c87-340b-4860-8af4-f6160941d465 None admin] PATH_INFO: `/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:59.789441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f8c87-340b-4860-8af4-f6160941d465 None admin] RBAC: Authorizing `identity:get_project(project_id=b9304b5154504dd6931ba78f3cf3ca5b)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:59.790870 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f8c87-340b-4860-8af4-f6160941d465 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:59.792531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 539/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:16:59 2022] GET /identity/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:16:59.798620 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a26e2-1c5a-484c-92cb-6b4467804bed None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:16:59.897134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a26e2-1c5a-484c-92cb-6b4467804bed None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:16:59.942292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24609c0-b9eb-4c03-a812-2db14881dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:16:59.942945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24609c0-b9eb-4c03-a812-2db14881dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:16:59.943785 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24609c0-b9eb-4c03-a812-2db14881dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:16:59.944057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24609c0-b9eb-4c03-a812-2db14881dbc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:16:59.944882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24609c0-b9eb-4c03-a812-2db14881dbc9 None admin] RBAC: Adding query filter params (role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=b9304b5154504dd6931ba78f3cf3ca5b, user.id=1fab6c73afd74b7faa1282da7b357659) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:16:59.945232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24609c0-b9eb-4c03-a812-2db14881dbc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:16:59.946660 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24609c0-b9eb-4c03-a812-2db14881dbc9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:16:59.951912 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 578/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:16:59 2022] GET /identity/v3/role_assignments?user.id=1fab6c73afd74b7faa1282da7b357659&scope.project.id=b9304b5154504dd6931ba78f3cf3ca5b&role.id=c8ee1507247e4b34a213a8033c46378f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:02.156742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb1c09-2d13-41d4-8d66-b4e323a98486 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:02.157313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb1c09-2d13-41d4-8d66-b4e323a98486 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:02.157313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb1c09-2d13-41d4-8d66-b4e323a98486 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:02.158718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 540/1118] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:02.164076 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511f22f-2098-4145-984a-7e2d02430e6f None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:02.164488 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511f22f-2098-4145-984a-7e2d02430e6f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:02.164619 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511f22f-2098-4145-984a-7e2d02430e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:02.181862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-5511f22f-2098-4145-984a-7e2d02430e6f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:02.183021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-5511f22f-2098-4145-984a-7e2d02430e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:02.253795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 579/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:02.276358 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb1c09-2d13-41d4-8d66-b4e323a98486 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:02.372169 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb1c09-2d13-41d4-8d66-b4e323a98486 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:02.403867 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac6d6a-25c1-493d-87d5-9b0c4eb9cedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:02.404700 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac6d6a-25c1-493d-87d5-9b0c4eb9cedd None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:02.404984 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac6d6a-25c1-493d-87d5-9b0c4eb9cedd None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:02.405295 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac6d6a-25c1-493d-87d5-9b0c4eb9cedd None admin] PATH_INFO: `/v3/roles/c8ee1507247e4b34a213a8033c46378f` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:02.406805 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac6d6a-25c1-493d-87d5-9b0c4eb9cedd None admin] RBAC: Authorizing `identity:get_role(role_id=c8ee1507247e4b34a213a8033c46378f)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:02.408116 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac6d6a-25c1-493d-87d5-9b0c4eb9cedd None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:02.409512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 541/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:02 2022] GET /identity/v3/roles/c8ee1507247e4b34a213a8033c46378f => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:02.415198 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511f22f-2098-4145-984a-7e2d02430e6f None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:02.507940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511f22f-2098-4145-984a-7e2d02430e6f None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:02.539460 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b2a1f-3f7f-448e-8258-58083530f2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:02.540314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2a1f-3f7f-448e-8258-58083530f2d5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:02.540590 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2a1f-3f7f-448e-8258-58083530f2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:02.540857 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2a1f-3f7f-448e-8258-58083530f2d5 None admin] PATH_INFO: `/v3/users/1fab6c73afd74b7faa1282da7b357659` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:02.542057 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b2a1f-3f7f-448e-8258-58083530f2d5 None admin] RBAC: Authorizing `identity:get_user(user_id=1fab6c73afd74b7faa1282da7b357659)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:02.543481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b2a1f-3f7f-448e-8258-58083530f2d5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:02.545108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 580/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:02 2022] GET /identity/v3/users/1fab6c73afd74b7faa1282da7b357659 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:02.550866 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac6d6a-25c1-493d-87d5-9b0c4eb9cedd None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:02.645051 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac6d6a-25c1-493d-87d5-9b0c4eb9cedd None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:02.676793 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e88114-d9eb-4094-be0e-e5294bc323c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:02.678142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88114-d9eb-4094-be0e-e5294bc323c2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:02.678430 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88114-d9eb-4094-be0e-e5294bc323c2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:02.678707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88114-d9eb-4094-be0e-e5294bc323c2 None admin] PATH_INFO: `/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:02.679886 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e88114-d9eb-4094-be0e-e5294bc323c2 None admin] RBAC: Authorizing `identity:get_project(project_id=b9304b5154504dd6931ba78f3cf3ca5b)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:02.681342 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e88114-d9eb-4094-be0e-e5294bc323c2 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:02.682999 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 542/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:17:02 2022] GET /identity/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:02.688647 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b2a1f-3f7f-448e-8258-58083530f2d5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:02.780171 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b2a1f-3f7f-448e-8258-58083530f2d5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:02.812408 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ad824-36ea-4d3d-8450-1b3f6186084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:02.813310 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad824-36ea-4d3d-8450-1b3f6186084f None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:02.813598 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad824-36ea-4d3d-8450-1b3f6186084f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:02.813873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ad824-36ea-4d3d-8450-1b3f6186084f None admin] PATH_INFO: `/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b/users/1fab6c73afd74b7faa1282da7b357659/roles/c8ee1507247e4b34a213a8033c46378f` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:02.816069 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ad824-36ea-4d3d-8450-1b3f6186084f None admin] RBAC: Authorizing `identity:create_grant(project_id=b9304b5154504dd6931ba78f3cf3ca5b, user_id=1fab6c73afd74b7faa1282da7b357659, role_id=c8ee1507247e4b34a213a8033c46378f)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:02.817520 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ad824-36ea-4d3d-8450-1b3f6186084f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:02.829538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 581/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:17:02 2022] PUT /identity/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b/users/1fab6c73afd74b7faa1282da7b357659/roles/c8ee1507247e4b34a213a8033c46378f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:17:05.027705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeec566-b627-4b8a-9f8f-913f166ad0dd None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:05.028245 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeec566-b627-4b8a-9f8f-913f166ad0dd None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:05.028245 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeec566-b627-4b8a-9f8f-913f166ad0dd None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:05.029656 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 543/1124] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:05.034808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17c972-b942-419a-89c7-8e52fc36279d None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:05.035232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17c972-b942-419a-89c7-8e52fc36279d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:05.035377 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17c972-b942-419a-89c7-8e52fc36279d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:05.051616 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-2d17c972-b942-419a-89c7-8e52fc36279d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:05.052796 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-2d17c972-b942-419a-89c7-8e52fc36279d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:05.115639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 582/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:05.139243 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeec566-b627-4b8a-9f8f-913f166ad0dd None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:05.234478 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeec566-b627-4b8a-9f8f-913f166ad0dd None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:05.267983 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4760e3e-a599-49d5-82d8-f36bd5bb52d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:05.268824 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4760e3e-a599-49d5-82d8-f36bd5bb52d4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:05.269138 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4760e3e-a599-49d5-82d8-f36bd5bb52d4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:05.269409 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4760e3e-a599-49d5-82d8-f36bd5bb52d4 None admin] PATH_INFO: `/v3/roles/c8ee1507247e4b34a213a8033c46378f` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:05.270865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4760e3e-a599-49d5-82d8-f36bd5bb52d4 None admin] RBAC: Authorizing `identity:get_role(role_id=c8ee1507247e4b34a213a8033c46378f)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:05.272246 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4760e3e-a599-49d5-82d8-f36bd5bb52d4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:05.273652 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 544/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:05 2022] GET /identity/v3/roles/c8ee1507247e4b34a213a8033c46378f => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:05.279546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17c972-b942-419a-89c7-8e52fc36279d None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:05.372019 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17c972-b942-419a-89c7-8e52fc36279d None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:05.403306 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32809c-819d-4004-bd67-3e9a857240d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:05.404157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32809c-819d-4004-bd67-3e9a857240d2 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:05.404432 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32809c-819d-4004-bd67-3e9a857240d2 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:05.404699 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32809c-819d-4004-bd67-3e9a857240d2 None admin] PATH_INFO: `/v3/users/1fab6c73afd74b7faa1282da7b357659` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:05.405899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32809c-819d-4004-bd67-3e9a857240d2 None admin] RBAC: Authorizing `identity:get_user(user_id=1fab6c73afd74b7faa1282da7b357659)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:05.407308 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32809c-819d-4004-bd67-3e9a857240d2 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:05.408888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 583/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:05 2022] GET /identity/v3/users/1fab6c73afd74b7faa1282da7b357659 => generated 286 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:05.414415 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4760e3e-a599-49d5-82d8-f36bd5bb52d4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:05.509293 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4760e3e-a599-49d5-82d8-f36bd5bb52d4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:05.540619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a86c-26d6-4412-a369-a3db721425f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:05.541219 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a86c-26d6-4412-a369-a3db721425f4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:05.541501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a86c-26d6-4412-a369-a3db721425f4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:05.541779 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a86c-26d6-4412-a369-a3db721425f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:05.605366 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a86c-26d6-4412-a369-a3db721425f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:05.606783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a86c-26d6-4412-a369-a3db721425f4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:05.698119 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 545/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:17:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:17:05.703987 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32809c-819d-4004-bd67-3e9a857240d2 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:05.803088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32809c-819d-4004-bd67-3e9a857240d2 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:05.835301 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a53d1-c81d-4b4d-8a38-96d48abf1856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:05.836142 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a53d1-c81d-4b4d-8a38-96d48abf1856 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:05.836423 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a53d1-c81d-4b4d-8a38-96d48abf1856 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:05.836697 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a53d1-c81d-4b4d-8a38-96d48abf1856 None admin] PATH_INFO: `/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:05.837890 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a53d1-c81d-4b4d-8a38-96d48abf1856 None admin] RBAC: Authorizing `identity:get_project(project_id=b9304b5154504dd6931ba78f3cf3ca5b)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:05.839312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a53d1-c81d-4b4d-8a38-96d48abf1856 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:05.840978 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 584/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:17:05 2022] GET /identity/v3/projects/b9304b5154504dd6931ba78f3cf3ca5b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:05.846945 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a86c-26d6-4412-a369-a3db721425f4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:05.939496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a86c-26d6-4412-a369-a3db721425f4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:05.971830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e5f93-7a56-4520-b270-8105c292a7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:05.972467 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e5f93-7a56-4520-b270-8105c292a7ef None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:05.972749 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e5f93-7a56-4520-b270-8105c292a7ef None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:05.973022 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e5f93-7a56-4520-b270-8105c292a7ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:05.973873 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e5f93-7a56-4520-b270-8105c292a7ef None admin] RBAC: Adding query filter params (role.id=c8ee1507247e4b34a213a8033c46378f, scope.project.id=b9304b5154504dd6931ba78f3cf3ca5b, user.id=1fab6c73afd74b7faa1282da7b357659) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:05.974769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e5f93-7a56-4520-b270-8105c292a7ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:05.976182 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e5f93-7a56-4520-b270-8105c292a7ef None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:05.981722 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 546/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:17:05 2022] GET /identity/v3/role_assignments?user.id=1fab6c73afd74b7faa1282da7b357659&scope.project.id=b9304b5154504dd6931ba78f3cf3ca5b&role.id=c8ee1507247e4b34a213a8033c46378f => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:08.198985 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5d151-d07b-4586-9ff7-58a25002054e None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:08.199533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5d151-d07b-4586-9ff7-58a25002054e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:08.199533 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5d151-d07b-4586-9ff7-58a25002054e None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:08.200917 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 585/1131] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:08.206166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9ec2-9bda-40a3-9000-b3346c171b68 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:08.206571 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9ec2-9bda-40a3-9000-b3346c171b68 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:08.206715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9ec2-9bda-40a3-9000-b3346c171b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:08.222878 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-adfa9ec2-9bda-40a3-9000-b3346c171b68 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:08.224034 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-adfa9ec2-9bda-40a3-9000-b3346c171b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:08.287534 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 547/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:08.310930 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5d151-d07b-4586-9ff7-58a25002054e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:08.407261 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5d151-d07b-4586-9ff7-58a25002054e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:08.438995 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76511249-47aa-4aa0-883e-9b544b7e1ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:08.439768 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76511249-47aa-4aa0-883e-9b544b7e1ac6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:08.440052 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76511249-47aa-4aa0-883e-9b544b7e1ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:08.440326 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76511249-47aa-4aa0-883e-9b544b7e1ac6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:08.441525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76511249-47aa-4aa0-883e-9b544b7e1ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:08.443529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76511249-47aa-4aa0-883e-9b544b7e1ac6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:08.445139 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 586/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:17:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:08.450866 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa9ec2-9bda-40a3-9000-b3346c171b68 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:08.543131 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa9ec2-9bda-40a3-9000-b3346c171b68 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:08.576024 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0244d3e6-b32a-4b8c-981e-19be3ab0c141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:08.576668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244d3e6-b32a-4b8c-981e-19be3ab0c141 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:08.576948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244d3e6-b32a-4b8c-981e-19be3ab0c141 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:08.577250 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244d3e6-b32a-4b8c-981e-19be3ab0c141 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:08.577956 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244d3e6-b32a-4b8c-981e-19be3ab0c141 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:08.579365 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0244d3e6-b32a-4b8c-981e-19be3ab0c141 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:08.604198 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 548/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:17:08 2022] POST /identity/v3/projects => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:17:10.837800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f3b0-72f6-4153-976d-ab04776bf956 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:10.838340 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f3b0-72f6-4153-976d-ab04776bf956 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:10.838340 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f3b0-72f6-4153-976d-ab04776bf956 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:10.839702 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 587/1135] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:10.844888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f072db0-79d9-4d17-bad1-c41190ac6f17 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:10.845274 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f072db0-79d9-4d17-bad1-c41190ac6f17 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:10.845488 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f072db0-79d9-4d17-bad1-c41190ac6f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:10.862242 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-6f072db0-79d9-4d17-bad1-c41190ac6f17 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:10.863417 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-6f072db0-79d9-4d17-bad1-c41190ac6f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:10.926170 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 549/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:10.950437 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424f3b0-72f6-4153-976d-ab04776bf956 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:11.045558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424f3b0-72f6-4153-976d-ab04776bf956 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:11.076694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52016017-ac70-4470-b236-35a9eaecff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:11.077484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016017-ac70-4470-b236-35a9eaecff57 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:11.077761 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016017-ac70-4470-b236-35a9eaecff57 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:11.078030 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016017-ac70-4470-b236-35a9eaecff57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:11.079180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52016017-ac70-4470-b236-35a9eaecff57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:11.080583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52016017-ac70-4470-b236-35a9eaecff57 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:11.086895 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 588/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:17:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:11.092597 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f072db0-79d9-4d17-bad1-c41190ac6f17 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:11.185931 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f072db0-79d9-4d17-bad1-c41190ac6f17 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:11.217544 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f27189-4782-4823-8e54-00857442039d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:11.218199 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f27189-4782-4823-8e54-00857442039d None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:11.218480 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f27189-4782-4823-8e54-00857442039d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:11.218753 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f27189-4782-4823-8e54-00857442039d None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:11.219458 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f27189-4782-4823-8e54-00857442039d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:11.220864 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f27189-4782-4823-8e54-00857442039d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:11.240908 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 550/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:17:11 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:17:13.466694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44640657-2397-4cc3-a05f-575a36a09de0 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:13.467225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44640657-2397-4cc3-a05f-575a36a09de0 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:13.467225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44640657-2397-4cc3-a05f-575a36a09de0 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:13.468592 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 589/1139] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:13.473770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5f07b-5238-48b9-9bcf-4959201dc116 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:13.474695 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5f07b-5238-48b9-9bcf-4959201dc116 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:13.474893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5f07b-5238-48b9-9bcf-4959201dc116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:13.490986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-97d5f07b-5238-48b9-9bcf-4959201dc116 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:13.492143 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-97d5f07b-5238-48b9-9bcf-4959201dc116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:13.554599 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 551/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:17:13.577235 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44640657-2397-4cc3-a05f-575a36a09de0 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:13.673137 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44640657-2397-4cc3-a05f-575a36a09de0 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:13.704163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90c15e-a217-44a8-be83-7fbc27e0c7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:13.704939 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90c15e-a217-44a8-be83-7fbc27e0c7ae None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:13.705244 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90c15e-a217-44a8-be83-7fbc27e0c7ae None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:13.705520 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90c15e-a217-44a8-be83-7fbc27e0c7ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:13.715592 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90c15e-a217-44a8-be83-7fbc27e0c7ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-3d90c15e-a217-44a8-be83-7fbc27e0c7ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:17:13.723175 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:17:13.724849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:17:13.725678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 590/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:17:13.730858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5f07b-5238-48b9-9bcf-4959201dc116 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:13.824299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5f07b-5238-48b9-9bcf-4959201dc116 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:13.855808 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:13.856460 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:13.856742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:13.857018 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:13.857787 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:13.858094 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:13.859452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:13.868044 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 552/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:13.873297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90c15e-a217-44a8-be83-7fbc27e0c7ae None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:13.967949 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90c15e-a217-44a8-be83-7fbc27e0c7ae None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:13.999692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0e2a3-c649-4e77-9f3f-3bde54ad4d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:14.000538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0e2a3-c649-4e77-9f3f-3bde54ad4d2f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:14.000816 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0e2a3-c649-4e77-9f3f-3bde54ad4d2f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:14.001111 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0e2a3-c649-4e77-9f3f-3bde54ad4d2f None admin] PATH_INFO: `/v3/users/3fb008b0bcfc4ea48d5b7535178147c0` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:14.013397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0e2a3-c649-4e77-9f3f-3bde54ad4d2f None admin] RBAC: Authorizing `identity:get_user(user_id=3fb008b0bcfc4ea48d5b7535178147c0)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:14.014825 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0e2a3-c649-4e77-9f3f-3bde54ad4d2f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:14.016444 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 591/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:13 2022] GET /identity/v3/users/3fb008b0bcfc4ea48d5b7535178147c0 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:14.022495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:14.116708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7450a12-524f-483e-af1d-67c6bf32718b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:14.148352 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67486eec-3226-4479-963a-1b1d90f3baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:14.148994 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67486eec-3226-4479-963a-1b1d90f3baf5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:14.149311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67486eec-3226-4479-963a-1b1d90f3baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:14.149588 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67486eec-3226-4479-963a-1b1d90f3baf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:14.212895 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67486eec-3226-4479-963a-1b1d90f3baf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:14.214364 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67486eec-3226-4479-963a-1b1d90f3baf5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:14.307234 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 553/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:17:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:17:14.313180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0e2a3-c649-4e77-9f3f-3bde54ad4d2f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:14.410013 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0e2a3-c649-4e77-9f3f-3bde54ad4d2f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:14.443095 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423163f6-60e6-4e2f-b190-f98197cd20f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:14.443954 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423163f6-60e6-4e2f-b190-f98197cd20f4 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:14.444237 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423163f6-60e6-4e2f-b190-f98197cd20f4 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:14.444511 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423163f6-60e6-4e2f-b190-f98197cd20f4 None admin] PATH_INFO: `/v3/projects/80c7307ee92a49e1a002a0fd5c826572` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:14.445720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423163f6-60e6-4e2f-b190-f98197cd20f4 None admin] RBAC: Authorizing `identity:get_project(project_id=80c7307ee92a49e1a002a0fd5c826572)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:14.447147 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423163f6-60e6-4e2f-b190-f98197cd20f4 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:14.454044 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 592/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:17:14 2022] GET /identity/v3/projects/80c7307ee92a49e1a002a0fd5c826572 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:14.455184 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67486eec-3226-4479-963a-1b1d90f3baf5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:14.553956 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67486eec-3226-4479-963a-1b1d90f3baf5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:14.587225 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31476a20-bcbf-4027-ada7-b5661e61e2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:14.587885 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31476a20-bcbf-4027-ada7-b5661e61e2ba None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:14.588166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31476a20-bcbf-4027-ada7-b5661e61e2ba None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:14.588441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31476a20-bcbf-4027-ada7-b5661e61e2ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:14.589291 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31476a20-bcbf-4027-ada7-b5661e61e2ba None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=80c7307ee92a49e1a002a0fd5c826572, user.id=3fb008b0bcfc4ea48d5b7535178147c0) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:14.589604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31476a20-bcbf-4027-ada7-b5661e61e2ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:14.591033 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31476a20-bcbf-4027-ada7-b5661e61e2ba None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:14.596327 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 554/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:17:14 2022] GET /identity/v3/role_assignments?user.id=3fb008b0bcfc4ea48d5b7535178147c0&scope.project.id=80c7307ee92a49e1a002a0fd5c826572&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:16.802596 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f2b8-1022-49e4-9a12-b89eb50910e0 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:16.803130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f2b8-1022-49e4-9a12-b89eb50910e0 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:16.803130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f2b8-1022-49e4-9a12-b89eb50910e0 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:16.804510 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 593/1147] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:16.809658 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21168d22-1d57-4d05-98a1-6f91369ffa6d None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:16.810076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21168d22-1d57-4d05-98a1-6f91369ffa6d None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:16.810201 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21168d22-1d57-4d05-98a1-6f91369ffa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:16.826901 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-21168d22-1d57-4d05-98a1-6f91369ffa6d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:16.828059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-21168d22-1d57-4d05-98a1-6f91369ffa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:16.890294 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 555/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:16.912726 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0f2b8-1022-49e4-9a12-b89eb50910e0 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:17.018645 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0f2b8-1022-49e4-9a12-b89eb50910e0 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:17.051130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6ff76-434e-4e3f-800b-17a4bf0f9717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:17.051934 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6ff76-434e-4e3f-800b-17a4bf0f9717 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:17.052218 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6ff76-434e-4e3f-800b-17a4bf0f9717 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:17.052490 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6ff76-434e-4e3f-800b-17a4bf0f9717 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:17.062187 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6ff76-434e-4e3f-800b-17a4bf0f9717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-cce6ff76-434e-4e3f-800b-17a4bf0f9717 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:17:17.070251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:17:17.071270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:17.072225 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:17:17.073191 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 594/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:17:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:17:17.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21168d22-1d57-4d05-98a1-6f91369ffa6d None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:17.170232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21168d22-1d57-4d05-98a1-6f91369ffa6d None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:17.202196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:17.202842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:17.203118 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:17.203389 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:17.204122 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:17.204422 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:17.205803 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:17.213362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 556/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:17:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:17.219027 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6ff76-434e-4e3f-800b-17a4bf0f9717 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:17.320721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6ff76-434e-4e3f-800b-17a4bf0f9717 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:17.351584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ba9c7-3589-4f29-93f3-70f471c0d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:17.352438 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ba9c7-3589-4f29-93f3-70f471c0d04e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:17.352717 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ba9c7-3589-4f29-93f3-70f471c0d04e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:17.352988 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ba9c7-3589-4f29-93f3-70f471c0d04e None admin] PATH_INFO: `/v3/users/3fb008b0bcfc4ea48d5b7535178147c0` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:17.354209 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ba9c7-3589-4f29-93f3-70f471c0d04e None admin] RBAC: Authorizing `identity:get_user(user_id=3fb008b0bcfc4ea48d5b7535178147c0)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:17.355648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ba9c7-3589-4f29-93f3-70f471c0d04e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:17.357288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 595/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:17 2022] GET /identity/v3/users/3fb008b0bcfc4ea48d5b7535178147c0 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:17.363034 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:17.456842 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d58b2-7702-4eec-8128-d72d6670beee None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:17.487354 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed78609-5d7f-49ec-beb0-1d93c595b8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:17.488197 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed78609-5d7f-49ec-beb0-1d93c595b8e9 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:17.488476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed78609-5d7f-49ec-beb0-1d93c595b8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:17.488746 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed78609-5d7f-49ec-beb0-1d93c595b8e9 None admin] PATH_INFO: `/v3/projects/80c7307ee92a49e1a002a0fd5c826572` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:17.490465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed78609-5d7f-49ec-beb0-1d93c595b8e9 None admin] RBAC: Authorizing `identity:get_project(project_id=80c7307ee92a49e1a002a0fd5c826572)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:17.491871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed78609-5d7f-49ec-beb0-1d93c595b8e9 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:17.493816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 557/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:17:17 2022] GET /identity/v3/projects/80c7307ee92a49e1a002a0fd5c826572 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:17.499466 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ba9c7-3589-4f29-93f3-70f471c0d04e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:17.595020 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ba9c7-3589-4f29-93f3-70f471c0d04e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:17.627736 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204a4d-9b42-48e8-804e-4b5ccc80b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:17.628614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204a4d-9b42-48e8-804e-4b5ccc80b46d None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:17.628896 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204a4d-9b42-48e8-804e-4b5ccc80b46d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:17.629203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204a4d-9b42-48e8-804e-4b5ccc80b46d None admin] PATH_INFO: `/v3/projects/80c7307ee92a49e1a002a0fd5c826572/users/3fb008b0bcfc4ea48d5b7535178147c0/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:17.631151 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204a4d-9b42-48e8-804e-4b5ccc80b46d None admin] RBAC: Authorizing `identity:create_grant(project_id=80c7307ee92a49e1a002a0fd5c826572, user_id=3fb008b0bcfc4ea48d5b7535178147c0, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:17.632578 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204a4d-9b42-48e8-804e-4b5ccc80b46d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:17.645030 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 596/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:17:17 2022] PUT /identity/v3/projects/80c7307ee92a49e1a002a0fd5c826572/users/3fb008b0bcfc4ea48d5b7535178147c0/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:17:19.845541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4cb30-1b72-411d-aa50-99fe27d981eb None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:19.846079 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4cb30-1b72-411d-aa50-99fe27d981eb None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:19.846079 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4cb30-1b72-411d-aa50-99fe27d981eb None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:19.847474 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 558/1154] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:19.852679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a91fb-b5c3-496e-95fc-8d9a0259b94c None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:19.853137 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a91fb-b5c3-496e-95fc-8d9a0259b94c None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:19.853259 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a91fb-b5c3-496e-95fc-8d9a0259b94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:19.869605 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-4e1a91fb-b5c3-496e-95fc-8d9a0259b94c None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:19.870764 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-4e1a91fb-b5c3-496e-95fc-8d9a0259b94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:19.935396 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 597/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:19.959240 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4cb30-1b72-411d-aa50-99fe27d981eb None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:20.055087 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4cb30-1b72-411d-aa50-99fe27d981eb None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:20.087601 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c747c-2220-4a06-899a-bbc9b6821cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:20.088373 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c747c-2220-4a06-899a-bbc9b6821cf6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:20.088657 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c747c-2220-4a06-899a-bbc9b6821cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:20.088934 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c747c-2220-4a06-899a-bbc9b6821cf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:20.098719 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c747c-2220-4a06-899a-bbc9b6821cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f27c747c-2220-4a06-899a-bbc9b6821cf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:17:20.106275 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:17:20.107286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:17:20.108088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 559/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:17:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:17:20.113313 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a91fb-b5c3-496e-95fc-8d9a0259b94c None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:20.206861 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a91fb-b5c3-496e-95fc-8d9a0259b94c None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:20.237584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:20.238227 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:20.238502 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:20.238773 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:20.240047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:20.240347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:20.241716 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:20.249523 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 598/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:17:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:20.255805 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c747c-2220-4a06-899a-bbc9b6821cf6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:20.349110 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c747c-2220-4a06-899a-bbc9b6821cf6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:20.382060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c517d0-5ca2-48c1-85ac-cf7325fcfa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:20.382907 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c517d0-5ca2-48c1-85ac-cf7325fcfa62 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:20.383190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c517d0-5ca2-48c1-85ac-cf7325fcfa62 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:20.383464 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c517d0-5ca2-48c1-85ac-cf7325fcfa62 None admin] PATH_INFO: `/v3/users/3fb008b0bcfc4ea48d5b7535178147c0` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:20.384643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c517d0-5ca2-48c1-85ac-cf7325fcfa62 None admin] RBAC: Authorizing `identity:get_user(user_id=3fb008b0bcfc4ea48d5b7535178147c0)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:20.386090 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c517d0-5ca2-48c1-85ac-cf7325fcfa62 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:20.387704 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 560/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:20 2022] GET /identity/v3/users/3fb008b0bcfc4ea48d5b7535178147c0 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:20.393412 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:20.488738 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5614fff-d919-49b3-9f7d-9d67671cd4ad None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:20.520495 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc6a55-efe7-4b0e-9b90-3c862da910e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:20.521148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc6a55-efe7-4b0e-9b90-3c862da910e9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:20.521427 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc6a55-efe7-4b0e-9b90-3c862da910e9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:20.521696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc6a55-efe7-4b0e-9b90-3c862da910e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:20.583046 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc6a55-efe7-4b0e-9b90-3c862da910e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:20.584484 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc6a55-efe7-4b0e-9b90-3c862da910e9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:20.674900 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 599/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:17:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:17:20.681328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c517d0-5ca2-48c1-85ac-cf7325fcfa62 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:20.773872 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c517d0-5ca2-48c1-85ac-cf7325fcfa62 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:20.805377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41b990-f125-4d99-8a0c-8e1dfda4e5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:20.806206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41b990-f125-4d99-8a0c-8e1dfda4e5a2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:20.806483 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41b990-f125-4d99-8a0c-8e1dfda4e5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:20.806753 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41b990-f125-4d99-8a0c-8e1dfda4e5a2 None admin] PATH_INFO: `/v3/projects/80c7307ee92a49e1a002a0fd5c826572` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:20.807913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41b990-f125-4d99-8a0c-8e1dfda4e5a2 None admin] RBAC: Authorizing `identity:get_project(project_id=80c7307ee92a49e1a002a0fd5c826572)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:20.809348 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41b990-f125-4d99-8a0c-8e1dfda4e5a2 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:20.810999 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 561/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:17:20 2022] GET /identity/v3/projects/80c7307ee92a49e1a002a0fd5c826572 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:20.816952 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc6a55-efe7-4b0e-9b90-3c862da910e9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:20.907849 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc6a55-efe7-4b0e-9b90-3c862da910e9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:20.939764 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1593c1-ac17-4144-81e5-3398cb1173bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:20.940343 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1593c1-ac17-4144-81e5-3398cb1173bd None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:20.940619 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1593c1-ac17-4144-81e5-3398cb1173bd None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:20.940890 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1593c1-ac17-4144-81e5-3398cb1173bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:20.941732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1593c1-ac17-4144-81e5-3398cb1173bd None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=80c7307ee92a49e1a002a0fd5c826572, user.id=3fb008b0bcfc4ea48d5b7535178147c0) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:20.942041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1593c1-ac17-4144-81e5-3398cb1173bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:20.943486 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1593c1-ac17-4144-81e5-3398cb1173bd None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:20.948914 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 600/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:17:20 2022] GET /identity/v3/role_assignments?user.id=3fb008b0bcfc4ea48d5b7535178147c0&scope.project.id=80c7307ee92a49e1a002a0fd5c826572&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 608 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:23.151541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949cf31-b1ef-4d6a-9fe6-5f1d3d9e77f4 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:23.152063 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949cf31-b1ef-4d6a-9fe6-5f1d3d9e77f4 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:23.152165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949cf31-b1ef-4d6a-9fe6-5f1d3d9e77f4 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:23.153495 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 562/1162] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:23.158727 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d8206-3d37-4267-b622-4ec3c5f48925 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:23.159088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d8206-3d37-4267-b622-4ec3c5f48925 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:23.159274 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d8206-3d37-4267-b622-4ec3c5f48925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:23.175845 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-d50d8206-3d37-4267-b622-4ec3c5f48925 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:23.177625 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-d50d8206-3d37-4267-b622-4ec3c5f48925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:23.242466 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 601/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:23.264394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949cf31-b1ef-4d6a-9fe6-5f1d3d9e77f4 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:23.359808 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949cf31-b1ef-4d6a-9fe6-5f1d3d9e77f4 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:23.390805 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c157a0-f3e3-4eff-b350-e1aa889fd70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:23.391430 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c157a0-f3e3-4eff-b350-e1aa889fd70a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:23.391708 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c157a0-f3e3-4eff-b350-e1aa889fd70a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:23.391977 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c157a0-f3e3-4eff-b350-e1aa889fd70a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:23.453408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c157a0-f3e3-4eff-b350-e1aa889fd70a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:23.454766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c157a0-f3e3-4eff-b350-e1aa889fd70a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:23.546382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 563/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:17:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:17:23.552260 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d8206-3d37-4267-b622-4ec3c5f48925 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:23.644607 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d8206-3d37-4267-b622-4ec3c5f48925 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:23.676171 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f75ae-7f78-416d-a280-069d1de72906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:23.676976 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f75ae-7f78-416d-a280-069d1de72906 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:23.677283 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f75ae-7f78-416d-a280-069d1de72906 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:23.677554 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f75ae-7f78-416d-a280-069d1de72906 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:23.683931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f75ae-7f78-416d-a280-069d1de72906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:23.685395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f75ae-7f78-416d-a280-069d1de72906 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-537f75ae-7f78-416d-a280-069d1de72906 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:17:23.699631 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:17:23.700733 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:17:23.701709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 602/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:17:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:17:23.706823 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c157a0-f3e3-4eff-b350-e1aa889fd70a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:23.799661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c157a0-f3e3-4eff-b350-e1aa889fd70a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:23.831279 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f5b60-aba9-4433-b84a-6ae80ca813e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:23.831926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f5b60-aba9-4433-b84a-6ae80ca813e5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:23.832207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f5b60-aba9-4433-b84a-6ae80ca813e5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:23.832481 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f5b60-aba9-4433-b84a-6ae80ca813e5 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:23.834008 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f5b60-aba9-4433-b84a-6ae80ca813e5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:23.834313 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f5b60-aba9-4433-b84a-6ae80ca813e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:23.835669 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f5b60-aba9-4433-b84a-6ae80ca813e5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:23.842498 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 564/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:17:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:26.042292 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62997bfd-d218-4105-b55d-3c109e29ac0b None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:26.042875 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62997bfd-d218-4105-b55d-3c109e29ac0b None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:26.042875 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62997bfd-d218-4105-b55d-3c109e29ac0b None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:26.044257 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 603/1167] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:26.049527 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5e314-2c2d-4be1-a22a-022455bb3f23 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:26.049957 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5e314-2c2d-4be1-a22a-022455bb3f23 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:26.050089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5e314-2c2d-4be1-a22a-022455bb3f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:26.066382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-add5e314-2c2d-4be1-a22a-022455bb3f23 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:26.068073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-add5e314-2c2d-4be1-a22a-022455bb3f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:26.131948 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 565/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:26.154558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62997bfd-d218-4105-b55d-3c109e29ac0b None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:26.251828 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62997bfd-d218-4105-b55d-3c109e29ac0b None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:26.284802 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412c349-baf9-463e-a020-b9cb17180ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:26.285482 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c349-baf9-463e-a020-b9cb17180ec7 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:26.285768 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c349-baf9-463e-a020-b9cb17180ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:26.286045 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c349-baf9-463e-a020-b9cb17180ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:26.286811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412c349-baf9-463e-a020-b9cb17180ec7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:26.288169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412c349-baf9-463e-a020-b9cb17180ec7 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:26.302913 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 604/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Mon May 9 15:17:26 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:17:28.537845 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf66f1-fbe4-41c3-911a-22801ecfb7e5 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:28.538342 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf66f1-fbe4-41c3-911a-22801ecfb7e5 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:28.538342 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf66f1-fbe4-41c3-911a-22801ecfb7e5 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:28.539692 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 566/1170] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:28.544919 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc456d5-5dd8-433a-ab1d-5bba2725cdde None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:28.545355 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc456d5-5dd8-433a-ab1d-5bba2725cdde None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:28.545639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc456d5-5dd8-433a-ab1d-5bba2725cdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:28.561916 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-3cc456d5-5dd8-433a-ab1d-5bba2725cdde None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:28.563087 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-3cc456d5-5dd8-433a-ab1d-5bba2725cdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:28.625610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 605/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:28.648769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf66f1-fbe4-41c3-911a-22801ecfb7e5 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:28.744012 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf66f1-fbe4-41c3-911a-22801ecfb7e5 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:28.775830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796e6b0-77b7-4c24-8606-cdb3865b1a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:28.776659 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796e6b0-77b7-4c24-8606-cdb3865b1a19 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:28.776942 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796e6b0-77b7-4c24-8606-cdb3865b1a19 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:28.777249 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796e6b0-77b7-4c24-8606-cdb3865b1a19 None admin] PATH_INFO: `/v3/domains/80a32f59aa554f93805197049eab35d6` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:28.787729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796e6b0-77b7-4c24-8606-cdb3865b1a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=80a32f59aa554f93805197049eab35d6)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:28.789242 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796e6b0-77b7-4c24-8606-cdb3865b1a19 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:28.790792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 567/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 9 15:17:28 2022] GET /identity/v3/domains/80a32f59aa554f93805197049eab35d6 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:28.796404 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc456d5-5dd8-433a-ab1d-5bba2725cdde None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:28.889958 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc456d5-5dd8-433a-ab1d-5bba2725cdde None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:28.921285 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3163e-55ec-4993-9774-3e481986165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:28.921929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3163e-55ec-4993-9774-3e481986165f None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:28.922203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3163e-55ec-4993-9774-3e481986165f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:28.922525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3163e-55ec-4993-9774-3e481986165f None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:28.923233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3163e-55ec-4993-9774-3e481986165f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:28.924676 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3163e-55ec-4993-9774-3e481986165f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:28.948409 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 606/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:17:28 2022] POST /identity/v3/projects => generated 361 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:17:31.317882 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f848fb-15d4-4be1-9140-7bb811c2e9fb None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:31.318413 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f848fb-15d4-4be1-9140-7bb811c2e9fb None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:31.318413 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f848fb-15d4-4be1-9140-7bb811c2e9fb None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:31.319789 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 568/1174] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:31.325091 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1e29-f979-4253-b026-328df9ac9b19 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:31.325590 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1e29-f979-4253-b026-328df9ac9b19 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:31.325688 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1e29-f979-4253-b026-328df9ac9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:31.342357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-4ffb1e29-f979-4253-b026-328df9ac9b19 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:31.343524 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-4ffb1e29-f979-4253-b026-328df9ac9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:31.406091 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 607/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:31.430802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f848fb-15d4-4be1-9140-7bb811c2e9fb None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:31.526998 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f848fb-15d4-4be1-9140-7bb811c2e9fb None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:31.559002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb8781-7656-4855-8b30-be3da802dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:31.559838 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb8781-7656-4855-8b30-be3da802dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:31.560120 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb8781-7656-4855-8b30-be3da802dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:31.560392 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb8781-7656-4855-8b30-be3da802dcb4 None admin] PATH_INFO: `/v3/domains/80a32f59aa554f93805197049eab35d6` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:31.561580 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb8781-7656-4855-8b30-be3da802dcb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=80a32f59aa554f93805197049eab35d6)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:31.562990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb8781-7656-4855-8b30-be3da802dcb4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:31.564564 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 569/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 9 15:17:31 2022] GET /identity/v3/domains/80a32f59aa554f93805197049eab35d6 => generated 267 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:31.570626 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1e29-f979-4253-b026-328df9ac9b19 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:31.661242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb1e29-f979-4253-b026-328df9ac9b19 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:31.692241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607280b-3e0e-4312-967b-ca2101279ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:31.693449 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607280b-3e0e-4312-967b-ca2101279ece None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:31.693728 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607280b-3e0e-4312-967b-ca2101279ece None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:31.693997 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607280b-3e0e-4312-967b-ca2101279ece None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:31.694688 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607280b-3e0e-4312-967b-ca2101279ece None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:31.696088 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607280b-3e0e-4312-967b-ca2101279ece None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:31.715317 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 608/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:17:31 2022] POST /identity/v3/users => generated 311 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:17:33.956371 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feef4b-ac98-4ef8-bb15-02c2472f71d9 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:33.956890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feef4b-ac98-4ef8-bb15-02c2472f71d9 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:33.956890 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feef4b-ac98-4ef8-bb15-02c2472f71d9 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:33.958337 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 570/1178] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:33.963620 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6f261-80d9-4022-8416-272ee701b31f None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:33.964043 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6f261-80d9-4022-8416-272ee701b31f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:33.964169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6f261-80d9-4022-8416-272ee701b31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:33.980350 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-ffe6f261-80d9-4022-8416-272ee701b31f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:33.981543 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-ffe6f261-80d9-4022-8416-272ee701b31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:34.043812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 609/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:34.066871 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04feef4b-ac98-4ef8-bb15-02c2472f71d9 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:34.162437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04feef4b-ac98-4ef8-bb15-02c2472f71d9 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:34.197856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee190f-220a-4cb4-866a-e98e4a4d3b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:34.198636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee190f-220a-4cb4-866a-e98e4a4d3b01 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:34.198915 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee190f-220a-4cb4-866a-e98e4a4d3b01 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:34.199185 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee190f-220a-4cb4-866a-e98e4a4d3b01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:34.208865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee190f-220a-4cb4-866a-e98e4a4d3b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b9ee190f-220a-4cb4-866a-e98e4a4d3b01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:17:34.216477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:17:34.217622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:17:34.218640 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 571/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:17:34.224177 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6f261-80d9-4022-8416-272ee701b31f None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:34.323315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6f261-80d9-4022-8416-272ee701b31f None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:34.354989 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:34.355640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:34.355919 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:34.356189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:34.356922 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:34.357253 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:34.358598 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:34.366744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 610/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:34.372851 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee190f-220a-4cb4-866a-e98e4a4d3b01 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:34.466450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee190f-220a-4cb4-866a-e98e4a4d3b01 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:34.497800 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74159137-5633-4d07-9fa5-f4f34c152f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:34.498642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159137-5633-4d07-9fa5-f4f34c152f5e None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:34.498916 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159137-5633-4d07-9fa5-f4f34c152f5e None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:34.499183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159137-5633-4d07-9fa5-f4f34c152f5e None admin] PATH_INFO: `/v3/users/594f8dffaf2940e68e2775f9334d2121` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:34.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74159137-5633-4d07-9fa5-f4f34c152f5e None admin] RBAC: Authorizing `identity:get_user(user_id=594f8dffaf2940e68e2775f9334d2121)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:34.513449 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74159137-5633-4d07-9fa5-f4f34c152f5e None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:34.515068 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 572/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:34 2022] GET /identity/v3/users/594f8dffaf2940e68e2775f9334d2121 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:34.521354 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:34.620120 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425fb86-8b4f-417d-ac2a-f677f9a4918b None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:34.676763 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e12277-3332-41d4-914a-ecd0b56e6ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:34.677605 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12277-3332-41d4-914a-ecd0b56e6ddc None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:34.677886 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12277-3332-41d4-914a-ecd0b56e6ddc None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:34.678157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12277-3332-41d4-914a-ecd0b56e6ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:34.740808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e12277-3332-41d4-914a-ecd0b56e6ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:34.742469 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e12277-3332-41d4-914a-ecd0b56e6ddc None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:34.832312 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 611/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:17:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:17:34.838849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74159137-5633-4d07-9fa5-f4f34c152f5e None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:34.931472 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74159137-5633-4d07-9fa5-f4f34c152f5e None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:34.968107 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e857b-9f3d-4bce-ba35-19741c7f335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:34.968967 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e857b-9f3d-4bce-ba35-19741c7f335f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:34.969277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e857b-9f3d-4bce-ba35-19741c7f335f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:34.969553 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e857b-9f3d-4bce-ba35-19741c7f335f None admin] PATH_INFO: `/v3/projects/7205cb5a39714b099089e5aac2bf733e` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:34.972982 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e857b-9f3d-4bce-ba35-19741c7f335f None admin] RBAC: Authorizing `identity:get_project(project_id=7205cb5a39714b099089e5aac2bf733e)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:34.974474 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e857b-9f3d-4bce-ba35-19741c7f335f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:34.976186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 573/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:17:34 2022] GET /identity/v3/projects/7205cb5a39714b099089e5aac2bf733e => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:34.982525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e12277-3332-41d4-914a-ecd0b56e6ddc None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:35.078158 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e12277-3332-41d4-914a-ecd0b56e6ddc None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:35.110075 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d0f3e-9557-45f9-9929-5b30599e9331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:35.110721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d0f3e-9557-45f9-9929-5b30599e9331 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:35.111005 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d0f3e-9557-45f9-9929-5b30599e9331 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:35.111282 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d0f3e-9557-45f9-9929-5b30599e9331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:35.112106 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d0f3e-9557-45f9-9929-5b30599e9331 None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=7205cb5a39714b099089e5aac2bf733e, user.id=594f8dffaf2940e68e2775f9334d2121) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:35.112419 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d0f3e-9557-45f9-9929-5b30599e9331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:35.113873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d0f3e-9557-45f9-9929-5b30599e9331 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:35.119375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 612/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:17:34 2022] GET /identity/v3/role_assignments?user.id=594f8dffaf2940e68e2775f9334d2121&scope.project.id=7205cb5a39714b099089e5aac2bf733e&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:37.295720 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc7166-eb3b-40bb-8de7-cd83bb6b5995 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:37.296191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc7166-eb3b-40bb-8de7-cd83bb6b5995 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:37.296283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc7166-eb3b-40bb-8de7-cd83bb6b5995 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:37.297690 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 574/1186] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:37.302921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35557135-441d-4def-9649-a8acc9721c47 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:37.303342 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35557135-441d-4def-9649-a8acc9721c47 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:37.303461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35557135-441d-4def-9649-a8acc9721c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:37.327374 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-35557135-441d-4def-9649-a8acc9721c47 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:37.328546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-35557135-441d-4def-9649-a8acc9721c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:37.390376 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 613/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:37.413109 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc7166-eb3b-40bb-8de7-cd83bb6b5995 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:37.509496 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc7166-eb3b-40bb-8de7-cd83bb6b5995 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:37.542086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093cceb-9fdc-4ed8-978e-690ec5cde3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:37.542862 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093cceb-9fdc-4ed8-978e-690ec5cde3ed None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:37.543145 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093cceb-9fdc-4ed8-978e-690ec5cde3ed None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:37.543424 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093cceb-9fdc-4ed8-978e-690ec5cde3ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:37.553210 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f093cceb-9fdc-4ed8-978e-690ec5cde3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f093cceb-9fdc-4ed8-978e-690ec5cde3ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:17:37.560742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:17:37.561822 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:17:37.562790 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 575/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:17:37.567837 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35557135-441d-4def-9649-a8acc9721c47 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:37.664258 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35557135-441d-4def-9649-a8acc9721c47 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:37.696041 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:37.696698 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:37.696984 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:37.697293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:37.698037 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:37.698347 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:37.699713 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:37.707448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 614/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:17:37.713806 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093cceb-9fdc-4ed8-978e-690ec5cde3ed None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:37.807533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093cceb-9fdc-4ed8-978e-690ec5cde3ed None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:37.839765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261f61a-a9c9-4c55-aee2-6a2adfbc9d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:37.840604 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261f61a-a9c9-4c55-aee2-6a2adfbc9d9d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:37.840886 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261f61a-a9c9-4c55-aee2-6a2adfbc9d9d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:37.841191 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261f61a-a9c9-4c55-aee2-6a2adfbc9d9d None admin] PATH_INFO: `/v3/users/594f8dffaf2940e68e2775f9334d2121` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:37.842372 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261f61a-a9c9-4c55-aee2-6a2adfbc9d9d None admin] RBAC: Authorizing `identity:get_user(user_id=594f8dffaf2940e68e2775f9334d2121)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:37.843785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261f61a-a9c9-4c55-aee2-6a2adfbc9d9d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:37.845409 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 576/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:37 2022] GET /identity/v3/users/594f8dffaf2940e68e2775f9334d2121 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:37.850975 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:37.943564 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa6fd5-1bd7-4343-9f0b-2cc92f58f93e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:37.975435 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a4602-d135-4c2b-8876-9c72f211e852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:37.976270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a4602-d135-4c2b-8876-9c72f211e852 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:37.976549 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a4602-d135-4c2b-8876-9c72f211e852 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:37.976819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a4602-d135-4c2b-8876-9c72f211e852 None admin] PATH_INFO: `/v3/projects/7205cb5a39714b099089e5aac2bf733e` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:37.978003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a4602-d135-4c2b-8876-9c72f211e852 None admin] RBAC: Authorizing `identity:get_project(project_id=7205cb5a39714b099089e5aac2bf733e)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:37.979407 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a4602-d135-4c2b-8876-9c72f211e852 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:37.981058 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 615/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:17:37 2022] GET /identity/v3/projects/7205cb5a39714b099089e5aac2bf733e => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:17:37.986863 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261f61a-a9c9-4c55-aee2-6a2adfbc9d9d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:38.084959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261f61a-a9c9-4c55-aee2-6a2adfbc9d9d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:38.115802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c352b-5bed-41d3-962e-697e0ae83346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:38.116676 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c352b-5bed-41d3-962e-697e0ae83346 None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:38.117237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c352b-5bed-41d3-962e-697e0ae83346 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:38.117512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c352b-5bed-41d3-962e-697e0ae83346 None admin] PATH_INFO: `/v3/projects/7205cb5a39714b099089e5aac2bf733e/users/594f8dffaf2940e68e2775f9334d2121/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:38.119463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c352b-5bed-41d3-962e-697e0ae83346 None admin] RBAC: Authorizing `identity:create_grant(project_id=7205cb5a39714b099089e5aac2bf733e, user_id=594f8dffaf2940e68e2775f9334d2121, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:38.120881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c352b-5bed-41d3-962e-697e0ae83346 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:38.133351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 577/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:17:37 2022] PUT /identity/v3/projects/7205cb5a39714b099089e5aac2bf733e/users/594f8dffaf2940e68e2775f9334d2121/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:17:40.317830 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e7467-ff1b-46be-8db8-bc31bf318ccd None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:40.318329 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e7467-ff1b-46be-8db8-bc31bf318ccd None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:40.318416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e7467-ff1b-46be-8db8-bc31bf318ccd None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:40.319760 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 616/1193] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:17:40.324856 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc187ef-c8c0-4bac-b39f-1a2cac88007e None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:40.325262 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc187ef-c8c0-4bac-b39f-1a2cac88007e None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:40.325450 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc187ef-c8c0-4bac-b39f-1a2cac88007e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:40.341644 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-ddc187ef-c8c0-4bac-b39f-1a2cac88007e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:17:40.342802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-ddc187ef-c8c0-4bac-b39f-1a2cac88007e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:17:40.405954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 578/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:17:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:17:40.429434 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e7467-ff1b-46be-8db8-bc31bf318ccd None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:40.526447 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e7467-ff1b-46be-8db8-bc31bf318ccd None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:40.557212 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0352-30ba-465d-ae63-cebf9bbb230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:40.558291 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0352-30ba-465d-ae63-cebf9bbb230d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:40.558577 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0352-30ba-465d-ae63-cebf9bbb230d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:40.558846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0352-30ba-465d-ae63-cebf9bbb230d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:40.568747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b0352-30ba-465d-ae63-cebf9bbb230d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-fe0b0352-30ba-465d-ae63-cebf9bbb230d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:17:40.576481 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:17:40.577584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:17:40.578397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 617/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:17:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:17:40.584050 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc187ef-c8c0-4bac-b39f-1a2cac88007e None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:40.675672 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc187ef-c8c0-4bac-b39f-1a2cac88007e None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:40.707046 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:40.708217 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:40.708494 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:40.708762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:40.709525 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:40.709829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:40.711177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:40.718703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 579/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:17:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:40.724803 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0352-30ba-465d-ae63-cebf9bbb230d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:40.816716 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0352-30ba-465d-ae63-cebf9bbb230d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:40.848820 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6745abd-ee9c-487c-85b4-143b1e0fbc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:40.849689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6745abd-ee9c-487c-85b4-143b1e0fbc09 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:40.849970 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6745abd-ee9c-487c-85b4-143b1e0fbc09 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:40.850246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6745abd-ee9c-487c-85b4-143b1e0fbc09 None admin] PATH_INFO: `/v3/users/594f8dffaf2940e68e2775f9334d2121` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:40.851427 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6745abd-ee9c-487c-85b4-143b1e0fbc09 None admin] RBAC: Authorizing `identity:get_user(user_id=594f8dffaf2940e68e2775f9334d2121)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:40.852854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6745abd-ee9c-487c-85b4-143b1e0fbc09 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:40.854471 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 618/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:17:40 2022] GET /identity/v3/users/594f8dffaf2940e68e2775f9334d2121 => generated 311 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:40.859986 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:40.955327 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d72be-3013-48f9-a89d-146e5c362193 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:40.987086 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20ad35-485e-4870-b23b-ff9985100229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:40.987715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20ad35-485e-4870-b23b-ff9985100229 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:40.987996 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20ad35-485e-4870-b23b-ff9985100229 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:40.988266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20ad35-485e-4870-b23b-ff9985100229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:41.051542 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20ad35-485e-4870-b23b-ff9985100229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:41.052967 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20ad35-485e-4870-b23b-ff9985100229 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:41.144479 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 580/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:17:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:17:41.150459 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6745abd-ee9c-487c-85b4-143b1e0fbc09 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:41.244819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6745abd-ee9c-487c-85b4-143b1e0fbc09 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:41.277143 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3449b4-ca21-4338-8f7f-7e31a7cd97b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:41.277989 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3449b4-ca21-4338-8f7f-7e31a7cd97b6 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:41.278269 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3449b4-ca21-4338-8f7f-7e31a7cd97b6 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:41.278542 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3449b4-ca21-4338-8f7f-7e31a7cd97b6 None admin] PATH_INFO: `/v3/projects/7205cb5a39714b099089e5aac2bf733e` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:41.279717 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3449b4-ca21-4338-8f7f-7e31a7cd97b6 None admin] RBAC: Authorizing `identity:get_project(project_id=7205cb5a39714b099089e5aac2bf733e)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:41.281165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3449b4-ca21-4338-8f7f-7e31a7cd97b6 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:41.282843 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 619/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:17:41 2022] GET /identity/v3/projects/7205cb5a39714b099089e5aac2bf733e => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:17:41.289472 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20ad35-485e-4870-b23b-ff9985100229 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:17:41.383258 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20ad35-485e-4870-b23b-ff9985100229 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:17:41.415783 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395455ba-5665-4e7e-a701-24ff08c28f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:17:41.416430 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395455ba-5665-4e7e-a701-24ff08c28f0c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:17:41.416710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395455ba-5665-4e7e-a701-24ff08c28f0c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:17:41.416983 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395455ba-5665-4e7e-a701-24ff08c28f0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:17:41.417833 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395455ba-5665-4e7e-a701-24ff08c28f0c None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=7205cb5a39714b099089e5aac2bf733e, user.id=594f8dffaf2940e68e2775f9334d2121) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:17:41.418142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395455ba-5665-4e7e-a701-24ff08c28f0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:17:41.419558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395455ba-5665-4e7e-a701-24ff08c28f0c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:17:41.425299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 581/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:17:41 2022] GET /identity/v3/role_assignments?user.id=594f8dffaf2940e68e2775f9334d2121&scope.project.id=7205cb5a39714b099089e5aac2bf733e&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:39.839142 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ba5a-e897-4f0e-84dc-eeb3e5a41b05 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:39.839700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ba5a-e897-4f0e-84dc-eeb3e5a41b05 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:39.839700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ba5a-e897-4f0e-84dc-eeb3e5a41b05 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:39.841150 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 620/1201] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:18:39.846913 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a4a0-e2d9-4aa9-9b92-3a7d0132491a None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:39.847382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a4a0-e2d9-4aa9-9b92-3a7d0132491a None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:39.847463 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a4a0-e2d9-4aa9-9b92-3a7d0132491a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:39.863990 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-a3e5a4a0-e2d9-4aa9-9b92-3a7d0132491a None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:18:39.865232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-a3e5a4a0-e2d9-4aa9-9b92-3a7d0132491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:18:39.939754 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 582/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:18:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:18:39.968033 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ba5a-e897-4f0e-84dc-eeb3e5a41b05 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:40.086895 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ba5a-e897-4f0e-84dc-eeb3e5a41b05 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:40.123627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:40.124346 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:40.124622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:40.124895 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:40.135174 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:40.137521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:40.158538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:40.160014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:40.165827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:40.165827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:40.165827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:40.165827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:40.165827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 621/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:40.166364 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a4a0-e2d9-4aa9-9b92-3a7d0132491a None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:40.299029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a4a0-e2d9-4aa9-9b92-3a7d0132491a None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:40.342829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006ddc4-3429-4328-b9ae-2e1252168b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:40.343881 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006ddc4-3429-4328-b9ae-2e1252168b9c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:40.344328 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006ddc4-3429-4328-b9ae-2e1252168b9c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:40.344652 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006ddc4-3429-4328-b9ae-2e1252168b9c None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:40.345409 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006ddc4-3429-4328-b9ae-2e1252168b9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:40.345716 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006ddc4-3429-4328-b9ae-2e1252168b9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:40.347943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006ddc4-3429-4328-b9ae-2e1252168b9c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:40.363013 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 583/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:40.369153 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:40.487474 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc72e32-a481-42cd-bd1a-d6720b64f6aa None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:40.533065 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310a960-4687-442e-85a5-6248de3f9e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:40.534330 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310a960-4687-442e-85a5-6248de3f9e96 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:40.534640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310a960-4687-442e-85a5-6248de3f9e96 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:40.535035 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310a960-4687-442e-85a5-6248de3f9e96 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:40.535751 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310a960-4687-442e-85a5-6248de3f9e96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:40.537692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310a960-4687-442e-85a5-6248de3f9e96 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:40.559846 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 622/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:18:40 2022] POST /identity/v3/users => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:18:42.898321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b0b99-4be7-41cb-ac81-71d0c0ece1ca None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:42.898884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b0b99-4be7-41cb-ac81-71d0c0ece1ca None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:42.898884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b0b99-4be7-41cb-ac81-71d0c0ece1ca None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:42.900452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 584/1206] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:18:42.905781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cdcf1-3476-4d41-9164-40e403c16e28 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:42.906194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cdcf1-3476-4d41-9164-40e403c16e28 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:42.906395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cdcf1-3476-4d41-9164-40e403c16e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:42.923931 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-073cdcf1-3476-4d41-9164-40e403c16e28 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:18:42.925185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-073cdcf1-3476-4d41-9164-40e403c16e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:18:42.996451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 623/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:18:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:18:43.019816 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b0b99-4be7-41cb-ac81-71d0c0ece1ca None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:43.129972 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b0b99-4be7-41cb-ac81-71d0c0ece1ca None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:43.169978 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1fbdf-81ba-445d-9e24-457272f5d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:43.170886 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1fbdf-81ba-445d-9e24-457272f5d561 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:43.171174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1fbdf-81ba-445d-9e24-457272f5d561 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:43.171476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1fbdf-81ba-445d-9e24-457272f5d561 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:43.183649 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1fbdf-81ba-445d-9e24-457272f5d561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-f9f1fbdf-81ba-445d-9e24-457272f5d561 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:18:43.193325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:43.194443 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:43.197920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 585/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:43.200618 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cdcf1-3476-4d41-9164-40e403c16e28 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:43.310768 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cdcf1-3476-4d41-9164-40e403c16e28 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:43.350335 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:43.350966 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:43.351250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:43.351527 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:43.355356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:43.355356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:43.355356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:43.362461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 624/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:18:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:43.369073 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1fbdf-81ba-445d-9e24-457272f5d561 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:43.485705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1fbdf-81ba-445d-9e24-457272f5d561 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:43.525561 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:43.526383 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:43.526735 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:43.526988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:43.538558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:43.540067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:43.558612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:43.559531 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:43.560385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:43.560385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:43.560385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:43.560385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:43.560385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 586/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:43.566221 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:43.673443 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b9e2a-0024-4ff3-8f02-28e7cd9d7207 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:43.711172 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:43.711872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:43.712155 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:43.712427 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:43.713170 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:43.713477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:43.714837 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:43.726867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 625/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:43.732756 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:43.848908 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092acec7-9846-44e6-a7cc-4eae4267414f None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:43.886964 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:43.887882 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:43.888268 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:43.888686 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] PATH_INFO: `/v3/users/placement` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:43.898001 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:43.899505 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:18:43.916207 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:43.917380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:43.918465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 587/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:43.923360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:44.041856 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cb254-7eae-41ed-aa14-178012b608bc None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:44.076425 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:44.077109 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:44.077397 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:44.077662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:44.078491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:44.078802 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:44.080231 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:44.098681 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 626/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:18:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:44.104099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:44.217384 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cd328-ba46-460b-8316-11a94eaf6adf None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:44.259493 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1599fa-8560-4690-a5c7-54a92ac5df74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:44.260284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1599fa-8560-4690-a5c7-54a92ac5df74 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:44.260687 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1599fa-8560-4690-a5c7-54a92ac5df74 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:44.261009 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1599fa-8560-4690-a5c7-54a92ac5df74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:44.343091 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1599fa-8560-4690-a5c7-54a92ac5df74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:44.344795 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1599fa-8560-4690-a5c7-54a92ac5df74 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:44.457305 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 588/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:18:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:18:44.463267 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:44.571827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c7452-fe35-41cb-8f3c-033e527d0a78 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:44.607571 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:44.608629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:44.608904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:44.609293 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:44.620629 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:44.622274 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:44.641594 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:44.642711 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:44.644358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:44.644358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:44.644358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:44.644358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:44.648365 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 627/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:44.649321 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1599fa-8560-4690-a5c7-54a92ac5df74 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:44.762557 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1599fa-8560-4690-a5c7-54a92ac5df74 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:44.804926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:44.805998 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:44.806512 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:44.806971 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:44.807811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:44.808123 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:44.809622 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:44.824590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 589/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:18:44.828543 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:44.941476 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2443-a4fa-4b23-ac0c-33ef4a6a285f None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:44.980813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:44.981662 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:44.981973 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:44.982288 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:44.992405 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:44.993938 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:18:45.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:45.009604 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:45.010448 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 628/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:18:45.015983 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:45.128076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a635d40-4eb4-4ce4-bf9b-5d251acce7fd None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:45.168192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb928280-53bf-4bcc-a7b6-bb7ae58db9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:45.168192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb928280-53bf-4bcc-a7b6-bb7ae58db9fa None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:45.168192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb928280-53bf-4bcc-a7b6-bb7ae58db9fa None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:45.168192 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb928280-53bf-4bcc-a7b6-bb7ae58db9fa None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:45.169441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb928280-53bf-4bcc-a7b6-bb7ae58db9fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:45.169652 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb928280-53bf-4bcc-a7b6-bb7ae58db9fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:45.171283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb928280-53bf-4bcc-a7b6-bb7ae58db9fa None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:45.185668 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 590/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:45.192786 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:45.302732 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d86491-3541-46c3-ad10-d4d3af4f0dcb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:45.341198 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c34a8-7a48-4626-ad19-2bf921e371c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:45.342356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c34a8-7a48-4626-ad19-2bf921e371c7 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:45.342641 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c34a8-7a48-4626-ad19-2bf921e371c7 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:45.342910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c34a8-7a48-4626-ad19-2bf921e371c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:45.343752 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c34a8-7a48-4626-ad19-2bf921e371c7 None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=c22d09c751714c30b4d1bb63e1323e8e) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:45.344070 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c34a8-7a48-4626-ad19-2bf921e371c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:45.345542 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c34a8-7a48-4626-ad19-2bf921e371c7 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:45.353548 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 629/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:18:45 2022] GET /identity/v3/role_assignments?user.id=c22d09c751714c30b4d1bb63e1323e8e&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:47.660261 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e98f4-d58b-49c2-8b02-58aa288e5df7 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:47.660804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e98f4-d58b-49c2-8b02-58aa288e5df7 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:47.660804 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e98f4-d58b-49c2-8b02-58aa288e5df7 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:47.662202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 591/1220] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:18:47.669118 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688e4f2-eb87-4f08-9c75-1d060e2fc79f None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:47.669118 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688e4f2-eb87-4f08-9c75-1d060e2fc79f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:47.669118 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688e4f2-eb87-4f08-9c75-1d060e2fc79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:47.687096 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-8688e4f2-eb87-4f08-9c75-1d060e2fc79f None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:18:47.688310 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-8688e4f2-eb87-4f08-9c75-1d060e2fc79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:18:47.767610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 630/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:18:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:18:47.792741 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e98f4-d58b-49c2-8b02-58aa288e5df7 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:47.906251 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e98f4-d58b-49c2-8b02-58aa288e5df7 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:47.946420 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82102fa3-de45-406a-a7d6-0ccf63390c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:47.947157 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82102fa3-de45-406a-a7d6-0ccf63390c35 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:47.947460 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82102fa3-de45-406a-a7d6-0ccf63390c35 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:47.947727 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82102fa3-de45-406a-a7d6-0ccf63390c35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:47.962229 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82102fa3-de45-406a-a7d6-0ccf63390c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-82102fa3-de45-406a-a7d6-0ccf63390c35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:18:47.971237 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:47.972159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:47.973016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 592/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:18:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:47.979679 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688e4f2-eb87-4f08-9c75-1d060e2fc79f None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:48.093895 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688e4f2-eb87-4f08-9c75-1d060e2fc79f None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:48.130811 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:48.131487 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:48.131770 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:48.132047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:48.132800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] RBAC: Adding query filter params (name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:48.133132 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:48.134525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:48.144451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 631/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:48.151016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82102fa3-de45-406a-a7d6-0ccf63390c35 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:48.261249 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82102fa3-de45-406a-a7d6-0ccf63390c35 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:48.296444 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:48.297394 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:48.297672 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:48.297997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:48.309920 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:48.311542 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:48.330062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:48.331517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:48.332335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:48.332335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:48.332335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:48.332335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:48.332335 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 593/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:48.338014 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:48.450525 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b15e53-d2ae-4d06-a817-577a7f071507 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:48.485902 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:48.486563 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:48.486840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:48.487110 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:48.487826 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:48.488128 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:48.489523 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:48.501477 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 632/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:48.508045 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:48.611992 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006e1ba-7273-4552-be91-708f95b5ea72 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:48.645270 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:48.646080 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:48.646399 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:48.646678 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] PATH_INFO: `/v3/users/placement` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:48.655167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:48.656620 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:18:48.671554 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:48.673031 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:48.673924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 594/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:18:48.679243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:48.781880 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e00a6-a79f-4796-899f-5d1a8174aa90 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:48.816840 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:48.817551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:48.817835 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:48.818110 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:48.818945 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:48.819252 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:48.820663 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:48.837860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 633/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:18:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:18:48.844286 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:48.946693 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513cac-2ed3-40ce-b28b-f6feedf22926 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:48.982385 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:48.983336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:48.983616 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:48.983910 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:48.994705 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:48.996195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:49.014266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:49.015355 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:49.016362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:49.016362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:49.016362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:49.016362 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:49.021357 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 595/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:49.022592 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:49.127016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c97af-90d6-433c-aee4-fe1ba18b87e5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:49.163228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:49.163967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:49.164246 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:49.164518 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:49.165449 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:49.165819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:49.167493 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:49.180119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 634/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:49.185821 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:49.291487 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a55a98-3b0d-4880-87b4-9f398efcf179 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:49.329726 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:49.330517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:49.330796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:49.331065 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:49.337699 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:49.339128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:18:49.353565 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:49.354612 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:49.355579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 596/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:18:49.361297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:49.472382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaa4a8-2fb7-4f43-ba6b-b7a39ee82bd5 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:49.510321 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407a93c-c9b1-43c4-8db1-f11888ab8e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:49.511105 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407a93c-c9b1-43c4-8db1-f11888ab8e6d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:49.511413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407a93c-c9b1-43c4-8db1-f11888ab8e6d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:49.511780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407a93c-c9b1-43c4-8db1-f11888ab8e6d None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:49.512648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407a93c-c9b1-43c4-8db1-f11888ab8e6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:49.512950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407a93c-c9b1-43c4-8db1-f11888ab8e6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:49.514701 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407a93c-c9b1-43c4-8db1-f11888ab8e6d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:49.528680 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 635/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:49.535766 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:49.647962 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1823a-6b65-4b6d-818b-818b35879d42 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:49.682427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5283b-01fe-4241-9498-42164fdf0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:49.683383 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5283b-01fe-4241-9498-42164fdf0eb6 None admin] REQUEST_METHOD: `PUT` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:49.683694 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5283b-01fe-4241-9498-42164fdf0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:49.683967 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5283b-01fe-4241-9498-42164fdf0eb6 None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/c22d09c751714c30b4d1bb63e1323e8e/roles/05908f87badf453aaf2ee23a431445f0` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:49.698575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5283b-01fe-4241-9498-42164fdf0eb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=c22d09c751714c30b4d1bb63e1323e8e, role_id=05908f87badf453aaf2ee23a431445f0)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:49.700190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5283b-01fe-4241-9498-42164fdf0eb6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:49.716457 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 597/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:18:49 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/c22d09c751714c30b4d1bb63e1323e8e/roles/05908f87badf453aaf2ee23a431445f0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:18:52.079841 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da80f0-4005-47e8-9b31-808657e48483 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:52.081092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da80f0-4005-47e8-9b31-808657e48483 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:52.081092 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da80f0-4005-47e8-9b31-808657e48483 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:52.086456 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 636/1233] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:18:52.087849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025619d9-bf84-451d-be10-0e423bd3950b None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:52.088312 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025619d9-bf84-451d-be10-0e423bd3950b None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:52.088444 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025619d9-bf84-451d-be10-0e423bd3950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:52.105894 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-025619d9-bf84-451d-be10-0e423bd3950b None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:18:52.107092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-025619d9-bf84-451d-be10-0e423bd3950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:18:52.182987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 598/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:18:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:18:52.209115 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da80f0-4005-47e8-9b31-808657e48483 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:52.316547 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da80f0-4005-47e8-9b31-808657e48483 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:52.349898 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4cc61-9d60-404a-8615-f66050f6b72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:52.350899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4cc61-9d60-404a-8615-f66050f6b72e None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:52.351290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4cc61-9d60-404a-8615-f66050f6b72e None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:52.351648 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4cc61-9d60-404a-8615-f66050f6b72e None admin] PATH_INFO: `/v3/roles/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:52.362744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4cc61-9d60-404a-8615-f66050f6b72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-d1f4cc61-9d60-404a-8615-f66050f6b72e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:18:52.370801 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:52.371860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:52.381387 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 637/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:18:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:52.382223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025619d9-bf84-451d-be10-0e423bd3950b None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:52.484377 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025619d9-bf84-451d-be10-0e423bd3950b None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:52.519236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:52.519909 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:52.520190 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:52.520465 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:52.521250 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] RBAC: Adding query filter params (name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:52.521551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:52.522914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:52.530713 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 599/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:18:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:52.540215 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4cc61-9d60-404a-8615-f66050f6b72e None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:52.645142 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4cc61-9d60-404a-8615-f66050f6b72e None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:52.678423 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:52.679222 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:52.679505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:52.679784 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:52.690894 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:52.692345 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:52.708865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:52.709872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:52.711124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:52.711124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:52.711124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:52.711124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:52.711124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 638/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:18:52.716037 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:52.822003 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6912d8-9630-4ad1-8d89-30b796ea787a None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:52.856590 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:52.857299 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:52.857585 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:52.857864 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:52.858584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:52.858893 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:52.860260 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:52.872628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 600/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:18:52.878643 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:52.988216 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e84dc8-26ba-4693-95f8-392b43ae1b38 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:53.021705 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:53.022507 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:53.022785 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:53.023054 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:53.030921 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:53.032836 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:18:53.051194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:53.052315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:53.058164 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 639/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:18:53.058861 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:53.167533 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475d0a7-3333-4f4f-ab83-3b5eb499d255 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:53.203482 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:53.204255 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:53.204521 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:53.204829 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:53.206347 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:53.206785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:53.208830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:53.226894 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 601/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:18:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:53.232873 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:53.338696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfa8dd-8a3f-457e-a075-54cb17379ce9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:53.375345 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bde85-f9fb-4066-aaca-069bd129fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:53.376004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bde85-f9fb-4066-aaca-069bd129fe4c None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:53.376290 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bde85-f9fb-4066-aaca-069bd129fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:53.376570 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bde85-f9fb-4066-aaca-069bd129fe4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:53.453417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bde85-f9fb-4066-aaca-069bd129fe4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:53.454868 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bde85-f9fb-4066-aaca-069bd129fe4c None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:53.556597 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 640/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:18:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:18:53.562482 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:53.668178 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cc03e-b5a1-4a8e-938b-abada0cb4bd1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:53.702540 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:53.703334 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:53.703615 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:53.703889 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:53.714714 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:53.716160 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:53.733769 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:53.734791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:53.735623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:53.735623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:53.735623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:53.735623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:53.735623 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 602/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:53.740929 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bde85-f9fb-4066-aaca-069bd129fe4c None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:53.844211 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bde85-f9fb-4066-aaca-069bd129fe4c None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:53.879226 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:53.879895 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:53.880180 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:53.880459 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:53.881243 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:53.881537 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:53.882918 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:53.897833 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 641/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:53.903452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:54.014281 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b3bc9-0d62-446d-b6c4-4c17512ae6d2 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:54.051888 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:54.052756 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:54.053076 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:54.053329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:54.060117 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:54.061617 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:18:54.075826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:54.076849 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:54.077926 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 603/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:18:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:54.083170 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:54.192275 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48ab34-fcc3-4d2b-9b00-74dd557acc84 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:54.225192 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae332d6-3778-4d0f-bba9-9933cfe9efda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:54.225979 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae332d6-3778-4d0f-bba9-9933cfe9efda None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:54.226264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae332d6-3778-4d0f-bba9-9933cfe9efda None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:54.226541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae332d6-3778-4d0f-bba9-9933cfe9efda None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:54.227341 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae332d6-3778-4d0f-bba9-9933cfe9efda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:54.227640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae332d6-3778-4d0f-bba9-9933cfe9efda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:54.229101 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae332d6-3778-4d0f-bba9-9933cfe9efda None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:54.242552 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 642/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:18:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:18:54.249722 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:54.360260 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e619b-8d8e-404c-a45b-89c14652a82b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:54.400419 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf09dd8-3db2-4bcb-b053-0b599dbc2e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:54.401324 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09dd8-3db2-4bcb-b053-0b599dbc2e65 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:54.401620 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09dd8-3db2-4bcb-b053-0b599dbc2e65 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:54.401897 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09dd8-3db2-4bcb-b053-0b599dbc2e65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:54.402927 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09dd8-3db2-4bcb-b053-0b599dbc2e65 None admin] RBAC: Adding query filter params (role.id=05908f87badf453aaf2ee23a431445f0, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=c22d09c751714c30b4d1bb63e1323e8e) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:54.403278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09dd8-3db2-4bcb-b053-0b599dbc2e65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:54.405281 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09dd8-3db2-4bcb-b053-0b599dbc2e65 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:54.412858 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 604/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:18:54 2022] GET /identity/v3/role_assignments?user.id=c22d09c751714c30b4d1bb63e1323e8e&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=05908f87badf453aaf2ee23a431445f0 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:56.802763 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe1a3e-74df-4f4c-9343-cd7b820567fd None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:56.803420 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe1a3e-74df-4f4c-9343-cd7b820567fd None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:56.803420 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe1a3e-74df-4f4c-9343-cd7b820567fd None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:56.804714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 643/1247] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:18:56.810021 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60140e8-691a-4ef5-9d5e-e6433a2a5d16 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:56.810447 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60140e8-691a-4ef5-9d5e-e6433a2a5d16 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:56.810541 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60140e8-691a-4ef5-9d5e-e6433a2a5d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:56.829477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-a60140e8-691a-4ef5-9d5e-e6433a2a5d16 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:18:56.831580 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-a60140e8-691a-4ef5-9d5e-e6433a2a5d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:18:56.905253 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 605/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:18:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:18:56.931107 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe1a3e-74df-4f4c-9343-cd7b820567fd None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:57.049640 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe1a3e-74df-4f4c-9343-cd7b820567fd None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:57.086903 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8536c-5885-462d-b9fa-9a45351106ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:57.087720 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8536c-5885-462d-b9fa-9a45351106ad None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:57.088023 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8536c-5885-462d-b9fa-9a45351106ad None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:57.088300 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8536c-5885-462d-b9fa-9a45351106ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:57.099653 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8536c-5885-462d-b9fa-9a45351106ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-16a8536c-5885-462d-b9fa-9a45351106ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:18:57.107382 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:57.108574 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:57.109806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 644/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:18:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:57.115245 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60140e8-691a-4ef5-9d5e-e6433a2a5d16 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:57.229973 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60140e8-691a-4ef5-9d5e-e6433a2a5d16 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:57.269130 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:57.269827 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:57.270091 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:57.270367 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:57.271125 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:57.271432 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:57.272802 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:57.282309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 606/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:18:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:18:57.288284 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8536c-5885-462d-b9fa-9a45351106ad None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:57.400746 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8536c-5885-462d-b9fa-9a45351106ad None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:57.436323 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:57.437304 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:57.437617 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:57.437894 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:57.450094 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:57.451583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:57.468800 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:57.469867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:57.471463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:57.471463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:57.471463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:57.471463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:57.471463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 645/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:57.476811 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:57.589757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f40e3-923a-4b3e-8b5b-22357fa34eb6 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:57.626775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:57.627462 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:57.627742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:57.628016 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:57.628765 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:57.629101 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:57.630461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:57.643981 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 607/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:57.650371 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:57.759514 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805388d-9203-4b02-8358-6ce2e5d05135 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:57.799315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:57.800670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:57.801224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:57.801716 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:57.813851 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:57.816314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:18:57.835795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:57.837205 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:18:57.839121 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:18:57.842194 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 646/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:57.843223 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:57.953480 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de7fa7-aa09-4c72-ace6-c01cf0316575 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:57.997841 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:57.998525 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:57.998885 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:57.999171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:58.000092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:58.000405 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:58.002172 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:58.023514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 608/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:18:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:18:58.029779 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:58.140417 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc521fb9-a869-41db-ab44-3a247e4653cb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:58.175016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3abbe-5ab5-47ac-b4a9-c6e568636b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:58.175668 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3abbe-5ab5-47ac-b4a9-c6e568636b6d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:58.175947 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3abbe-5ab5-47ac-b4a9-c6e568636b6d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:58.176222 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3abbe-5ab5-47ac-b4a9-c6e568636b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:58.247584 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3abbe-5ab5-47ac-b4a9-c6e568636b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:58.249576 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3abbe-5ab5-47ac-b4a9-c6e568636b6d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:58.356084 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 647/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:18:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:18:58.362167 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:58.463710 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56c715-927a-4a71-8215-a456aae3805c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:58.499386 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:58.500309 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:58.500739 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:58.501067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:58.512691 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:58.514484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:18:58.534284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:58.535177 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:58.536142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:18:58.536142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:18:58.536142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:18:58.536142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:58.536142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 609/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:58.541586 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3abbe-5ab5-47ac-b4a9-c6e568636b6d None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:58.647943 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3abbe-5ab5-47ac-b4a9-c6e568636b6d None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:58.684678 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:58.685375 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:58.685652 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:58.685923 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:58.686642 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:58.686939 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:58.688297 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:58.700472 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 648/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:18:58.706283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:58.821351 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b10af-309f-4422-a56f-448a768fd097 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:58.857641 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:58.858437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:58.858716 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:58.858989 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:58.866995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:58.868753 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:18:58.885865 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:18:58.886898 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:18:58.888020 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 610/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:18:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:18:58.894306 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:59.016016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c837da-667d-4613-929a-bb577a8be385 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:59.055902 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84871171-9fea-405e-b4fd-a14c5e6c7895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:59.056569 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84871171-9fea-405e-b4fd-a14c5e6c7895 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:59.056854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84871171-9fea-405e-b4fd-a14c5e6c7895 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:59.057154 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84871171-9fea-405e-b4fd-a14c5e6c7895 None admin] PATH_INFO: `/v3/projects` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:59.057941 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84871171-9fea-405e-b4fd-a14c5e6c7895 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:59.058251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84871171-9fea-405e-b4fd-a14c5e6c7895 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:59.059682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84871171-9fea-405e-b4fd-a14c5e6c7895 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:59.072045 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 649/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:18:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:18:59.078131 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:18:59.184002 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c38b43-3f97-41de-93b1-23110e16fe9c None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:18:59.217649 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151df97-287e-4b3a-b1ac-8d5249e5f6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:18:59.218390 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151df97-287e-4b3a-b1ac-8d5249e5f6dc None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:18:59.218661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151df97-287e-4b3a-b1ac-8d5249e5f6dc None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:18:59.218932 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151df97-287e-4b3a-b1ac-8d5249e5f6dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:18:59.219753 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151df97-287e-4b3a-b1ac-8d5249e5f6dc None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=c22d09c751714c30b4d1bb63e1323e8e) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:18:59.220059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151df97-287e-4b3a-b1ac-8d5249e5f6dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:18:59.222143 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151df97-287e-4b3a-b1ac-8d5249e5f6dc None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:18:59.227746 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 611/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:18:59 2022] GET /identity/v3/role_assignments?user.id=c22d09c751714c30b4d1bb63e1323e8e&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:01.416146 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8522-aadf-4c3f-a084-2f7b6ef523c2 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:01.416674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8522-aadf-4c3f-a084-2f7b6ef523c2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:01.416674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8522-aadf-4c3f-a084-2f7b6ef523c2 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:01.418089 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 650/1261] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:19:01.423246 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158a309-36b6-4c6e-9a0e-a09e50b10ecb None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:01.424176 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158a309-36b6-4c6e-9a0e-a09e50b10ecb None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:01.424311 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158a309-36b6-4c6e-9a0e-a09e50b10ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:01.441221 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-8158a309-36b6-4c6e-9a0e-a09e50b10ecb None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:19:01.442400 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-8158a309-36b6-4c6e-9a0e-a09e50b10ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:19:01.505274 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 612/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:19:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:19:01.526812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8522-aadf-4c3f-a084-2f7b6ef523c2 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:01.620841 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8522-aadf-4c3f-a084-2f7b6ef523c2 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:01.652226 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b04f5-ba34-422a-9429-ee813dafbbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:01.652985 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b04f5-ba34-422a-9429-ee813dafbbb9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:01.653291 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b04f5-ba34-422a-9429-ee813dafbbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:01.653562 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b04f5-ba34-422a-9429-ee813dafbbb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:01.663059 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b04f5-ba34-422a-9429-ee813dafbbb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-813b04f5-ba34-422a-9429-ee813dafbbb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:19:01.670593 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:01.671595 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:19:01.672706 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 651/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:01.677997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158a309-36b6-4c6e-9a0e-a09e50b10ecb None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:01.771427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158a309-36b6-4c6e-9a0e-a09e50b10ecb None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:01.802830 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:01.803420 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:01.803697 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] PATH_INFO: `/v3/roles` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:01.804702 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:01.805005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:01.806382 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:01.813929 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 613/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:19:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:01.820433 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b04f5-ba34-422a-9429-ee813dafbbb9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:01.933882 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b04f5-ba34-422a-9429-ee813dafbbb9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:01.969426 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:01.971076 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:01.971388 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:01.971808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:01.984496 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:01.986995 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:19:02.006963 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:02.007977 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:02.009022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:19:02.009022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:19:02.009022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:19:02.009022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:19:02.009022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 652/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:02.013914 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:02.108751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956928b0-a9f7-4301-9291-4ec21342cd3b None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:02.140585 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:02.141266 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:02.141551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:02.141824 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:02.142628 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:02.142934 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:02.144320 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:02.155477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 614/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:02.161538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:02.259814 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679209-c97c-4c85-9fc6-efcb75f09118 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:02.292436 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:02.293256 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:02.293538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:02.293813 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] PATH_INFO: `/v3/users/placement` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:02.301062 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:02.302503 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:19:02.316386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:02.317406 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:19:02.318359 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 653/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:19:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:02.323426 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:02.417645 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece9f16-1c35-4a30-be33-c61afad3d2ed None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:02.449826 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:02.450484 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:02.450762 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:02.451039 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] PATH_INFO: `/v3/users` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:02.451860 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:02.452166 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:02.453615 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:02.469595 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 615/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:19:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:02.475230 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:02.572463 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53197ab-d175-45a1-adb8-5b741ccfbd46 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:02.604551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:02.605356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:02.605639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:02.605912 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:02.616726 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:02.618189 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:19:02.636250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:02.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:02.638521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:19:02.638521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:19:02.638521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:19:02.638521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:19:02.638521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 654/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:02.643123 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:02.735502 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52a55-bd25-4c7c-bcb4-c92026123c02 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:02.767503 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:02.768150 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:02.768431 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:02.768703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:02.769442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:02.769749 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:02.771099 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:02.781636 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 616/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:19:02.787602 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:02.883163 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88c43a-9fd4-405c-a8e7-35d90beb8ae9 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:02.914048 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:02.914822 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:02.915100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:02.915372 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:02.921650 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:02.923066 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:19:02.938001 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:02.938967 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:19:02.940385 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 655/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:19:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:02.946008 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:03.049954 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655e077-fe2c-494c-9dd4-f0a37c68e57d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:03.081364 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64697367-510d-42e1-87c5-817e8a8cdaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:03.082023 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64697367-510d-42e1-87c5-817e8a8cdaae None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:03.082304 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64697367-510d-42e1-87c5-817e8a8cdaae None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:03.082579 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64697367-510d-42e1-87c5-817e8a8cdaae None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:03.083354 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64697367-510d-42e1-87c5-817e8a8cdaae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:03.083661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64697367-510d-42e1-87c5-817e8a8cdaae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:03.085097 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64697367-510d-42e1-87c5-817e8a8cdaae None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:03.096501 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 617/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:03.102428 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:03.195673 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f6aa5-d34a-42c8-bd97-5bae948f2deb None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:03.227845 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8c1cd-7a38-4f53-b3f5-4ffb7caeff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:03.227845 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8c1cd-7a38-4f53-b3f5-4ffb7caeff7d None admin] REQUEST_METHOD: `PUT` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:03.228233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8c1cd-7a38-4f53-b3f5-4ffb7caeff7d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:03.228233 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8c1cd-7a38-4f53-b3f5-4ffb7caeff7d None admin] PATH_INFO: `/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/c22d09c751714c30b4d1bb63e1323e8e/roles/a48318292c764734942e1b10b9fe9ea7` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:03.230404 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8c1cd-7a38-4f53-b3f5-4ffb7caeff7d None admin] RBAC: Authorizing `identity:create_grant(project_id=5861448983e849eaa5ee1cc05ffc2694, user_id=c22d09c751714c30b4d1bb63e1323e8e, role_id=a48318292c764734942e1b10b9fe9ea7)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:03.231827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8c1cd-7a38-4f53-b3f5-4ffb7caeff7d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:03.245742 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 656/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:19:03 2022] PUT /identity/v3/projects/5861448983e849eaa5ee1cc05ffc2694/users/c22d09c751714c30b4d1bb63e1323e8e/roles/a48318292c764734942e1b10b9fe9ea7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:19:05.441729 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd17e1-28c2-4fc3-a86e-8a1ae14388ac None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:05.442235 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd17e1-28c2-4fc3-a86e-8a1ae14388ac None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:05.442336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd17e1-28c2-4fc3-a86e-8a1ae14388ac None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:05.443649 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 618/1274] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:19:05.448784 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff503-3cf3-4245-a95e-26a25fa08728 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:05.449239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff503-3cf3-4245-a95e-26a25fa08728 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:05.449372 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff503-3cf3-4245-a95e-26a25fa08728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:05.466148 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-77bff503-3cf3-4245-a95e-26a25fa08728 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:19:05.467360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-77bff503-3cf3-4245-a95e-26a25fa08728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:19:05.538808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 657/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:19:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:19:05.561731 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd17e1-28c2-4fc3-a86e-8a1ae14388ac None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:05.656472 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd17e1-28c2-4fc3-a86e-8a1ae14388ac None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:05.688141 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7246475-6037-4b07-8415-0b2ad0f1bc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:05.688905 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246475-6037-4b07-8415-0b2ad0f1bc7d None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:05.689716 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246475-6037-4b07-8415-0b2ad0f1bc7d None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:05.689988 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246475-6037-4b07-8415-0b2ad0f1bc7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:05.699522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7246475-6037-4b07-8415-0b2ad0f1bc7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-b7246475-6037-4b07-8415-0b2ad0f1bc7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:19:05.707584 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:05.708611 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:05.709602 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 619/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:05.714625 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bff503-3cf3-4245-a95e-26a25fa08728 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:05.809114 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bff503-3cf3-4245-a95e-26a25fa08728 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:05.841744 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:05.842401 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:05.842682 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:05.842955 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] PATH_INFO: `/v3/roles` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:05.843695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] RBAC: Adding query filter params (name=admin) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:05.844000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:05.845386 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:05.853022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 658/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:05.858654 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7246475-6037-4b07-8415-0b2ad0f1bc7d None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:05.959446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7246475-6037-4b07-8415-0b2ad0f1bc7d None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:05.997823 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:05.998812 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:05.999205 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:05.999607 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:06.012001 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:06.013642 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:19:06.030707 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:06.031715 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:06.032752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:19:06.032752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:19:06.032752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:19:06.032752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:06.032752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 620/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:06.037861 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:06.131714 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30d5b-e4ab-4b64-bc4b-f15756dcd21a None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:06.163979 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:06.164703 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:06.165063 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:06.165336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] PATH_INFO: `/v3/domains` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:06.166045 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] RBAC: Adding query filter params (name=Default) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:06.166410 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:06.167865 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:06.178933 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 659/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:06.184436 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:06.278388 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee1a3b-394b-415a-adfb-39b489722214 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:06.309927 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:06.310709 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:06.310985 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:06.311254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:06.318868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:06.320284 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:19:06.334718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:06.335643 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:06.336661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 621/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:19:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:06.341636 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:06.435047 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca5363-b7ea-4da8-b019-0f723e87ccff None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:06.466854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:06.467504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:06.467780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:06.468050 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] PATH_INFO: `/v3/users` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:06.468867 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:06.469203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:06.470670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:06.486701 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 660/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:19:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:06.492558 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:06.585880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2073ee-a39e-4478-a8f4-20cc3a2312a4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:06.616896 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c0341-b7e4-498c-8f43-2868b18b9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:06.617552 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c0341-b7e4-498c-8f43-2868b18b9537 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:06.617831 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c0341-b7e4-498c-8f43-2868b18b9537 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:06.618102 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c0341-b7e4-498c-8f43-2868b18b9537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:06.681206 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c0341-b7e4-498c-8f43-2868b18b9537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:06.682630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c0341-b7e4-498c-8f43-2868b18b9537 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:06.774592 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 622/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:19:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:19:06.780228 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:06.882792 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9437a-a63b-4918-9fad-49d96679b109 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:06.914622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:06.915394 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:06.915672 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:06.915941 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:06.925344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:06.926765 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:19:06.946344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:06.947357 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:06.948239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:19:06.948239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:19:06.948239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:19:06.948239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:19:06.948239 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 661/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:06.954427 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c0341-b7e4-498c-8f43-2868b18b9537 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:07.050619 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c0341-b7e4-498c-8f43-2868b18b9537 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:07.083197 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:07.083848 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:07.084133 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:07.084410 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] PATH_INFO: `/v3/domains` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:07.085150 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:07.085458 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:07.086828 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:07.097991 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 623/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:07.103603 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:07.208910 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007b54b-d16e-4f50-851e-36e5de73bc50 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:07.246187 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:07.246980 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:07.247264 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:07.247539 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] PATH_INFO: `/v3/projects/service` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:07.254220 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:07.255666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:19:07.269049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application with Lock( May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:07.270003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:07.271022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:19:07.271022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:19:07.271022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:19:07.271022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:19:07.271022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:19:07.271022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:19:07.271022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:19:07.271022 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: ERROR keystone.server.flask.application May 09 15:19:07.273188 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 662/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:19:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:07.276014 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:07.371174 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa37da4-f868-43d5-8000-2069a9586e42 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:07.403104 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222de6c-2df1-4538-a26b-cc8d690afe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:07.403757 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222de6c-2df1-4538-a26b-cc8d690afe38 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:07.404042 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222de6c-2df1-4538-a26b-cc8d690afe38 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:07.404317 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222de6c-2df1-4538-a26b-cc8d690afe38 None admin] PATH_INFO: `/v3/projects` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:07.405128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222de6c-2df1-4538-a26b-cc8d690afe38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:07.405437 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222de6c-2df1-4538-a26b-cc8d690afe38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:07.406855 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222de6c-2df1-4538-a26b-cc8d690afe38 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:07.418775 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 624/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:19:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:07.425459 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:07.534440 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaadba-8f72-415d-ab98-243409ae2167 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:07.565350 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe8f9c-2b46-43c7-bdf9-d4205fb6c80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:07.565990 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8f9c-2b46-43c7-bdf9-d4205fb6c80d None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:07.566267 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8f9c-2b46-43c7-bdf9-d4205fb6c80d None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:07.566536 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8f9c-2b46-43c7-bdf9-d4205fb6c80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:07.567352 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe8f9c-2b46-43c7-bdf9-d4205fb6c80d None admin] RBAC: Adding query filter params (role.id=a48318292c764734942e1b10b9fe9ea7, scope.project.id=5861448983e849eaa5ee1cc05ffc2694, user.id=c22d09c751714c30b4d1bb63e1323e8e) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:07.567657 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe8f9c-2b46-43c7-bdf9-d4205fb6c80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:07.569095 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe8f9c-2b46-43c7-bdf9-d4205fb6c80d None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:07.574622 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 663/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:19:07 2022] GET /identity/v3/role_assignments?user.id=c22d09c751714c30b4d1bb63e1323e8e&scope.project.id=5861448983e849eaa5ee1cc05ffc2694&role.id=a48318292c764734942e1b10b9fe9ea7 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:09.819232 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b227-fbd5-4c33-9421-aa9aae956db3 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:09.819785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b227-fbd5-4c33-9421-aa9aae956db3 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:09.819785 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b227-fbd5-4c33-9421-aa9aae956db3 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:09.821204 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 625/1288] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:19:09.826608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07047d67-97ad-463f-8159-fd06483839c2 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:09.827016 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07047d67-97ad-463f-8159-fd06483839c2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:09.827165 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07047d67-97ad-463f-8159-fd06483839c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:09.843360 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-07047d67-97ad-463f-8159-fd06483839c2 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:19:09.844558 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-07047d67-97ad-463f-8159-fd06483839c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:19:09.908224 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 664/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:19:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:19:09.930375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0b227-fbd5-4c33-9421-aa9aae956db3 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:10.038029 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0b227-fbd5-4c33-9421-aa9aae956db3 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:10.070435 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:10.071159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:10.071439 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:10.071712 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:10.072357 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:10.073737 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:10.085215 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:10.086381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:10.087375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 626/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:19:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:10.092646 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07047d67-97ad-463f-8159-fd06483839c2 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:10.186695 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07047d67-97ad-463f-8159-fd06483839c2 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:10.218256 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aac961-f5ab-4470-addf-482240f8ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:10.218912 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aac961-f5ab-4470-addf-482240f8ec14 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:10.219190 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aac961-f5ab-4470-addf-482240f8ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:10.219462 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aac961-f5ab-4470-addf-482240f8ec14 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:10.220700 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aac961-f5ab-4470-addf-482240f8ec14 None admin] RBAC: Adding query filter params (name=placement) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:10.221003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aac961-f5ab-4470-addf-482240f8ec14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:10.222379 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aac961-f5ab-4470-addf-482240f8ec14 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:10.226965 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 665/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:19:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:10.232661 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:10.326851 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a3bcf-6b9a-40ba-94fc-d449d8a606b1 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:10.359222 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c47530-f890-49bb-a632-a558ba1b99bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:10.359880 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47530-f890-49bb-a632-a558ba1b99bf None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:10.360164 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47530-f890-49bb-a632-a558ba1b99bf None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:10.360441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c47530-f890-49bb-a632-a558ba1b99bf None admin] PATH_INFO: `/v3/services` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:10.361770 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c47530-f890-49bb-a632-a558ba1b99bf None admin] RBAC: Adding query filter params (type=placement) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:10.362110 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c47530-f890-49bb-a632-a558ba1b99bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:10.363477 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c47530-f890-49bb-a632-a558ba1b99bf None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:10.368111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 627/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:19:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:12.537356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d228-7e0c-4220-9f0e-ac7c55c430b9 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:12.537860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d228-7e0c-4220-9f0e-ac7c55c430b9 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:12.537860 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d228-7e0c-4220-9f0e-ac7c55c430b9 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:12.539241 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 666/1293] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:19:12.544400 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c1743-66b5-4c86-921f-fd98be8998bb None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:12.544806 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c1743-66b5-4c86-921f-fd98be8998bb None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:12.544949 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c1743-66b5-4c86-921f-fd98be8998bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:12.563195 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-288c1743-66b5-4c86-921f-fd98be8998bb None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:19:12.565448 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-288c1743-66b5-4c86-921f-fd98be8998bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:19:12.631057 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 628/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:19:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:19:12.653583 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6d228-7e0c-4220-9f0e-ac7c55c430b9 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:12.747358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6d228-7e0c-4220-9f0e-ac7c55c430b9 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:12.778687 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5082107-65ed-49d4-b563-97132ce86911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:12.779323 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5082107-65ed-49d4-b563-97132ce86911 None admin] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:12.779601 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5082107-65ed-49d4-b563-97132ce86911 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:12.779872 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5082107-65ed-49d4-b563-97132ce86911 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:12.780578 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5082107-65ed-49d4-b563-97132ce86911 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:12.781952 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5082107-65ed-49d4-b563-97132ce86911 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:12.794056 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 667/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:19:12 2022] POST /identity/v3/services => generated 246 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:19:15.045946 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4356e65-4487-49f8-a4f0-198d33da1fd0 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:15.046492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4356e65-4487-49f8-a4f0-198d33da1fd0 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:15.046492 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4356e65-4487-49f8-a4f0-198d33da1fd0 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:15.047866 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 629/1296] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:19:15.053827 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd8dc3-aead-4455-8986-e0495a1a15b8 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:15.054206 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd8dc3-aead-4455-8986-e0495a1a15b8 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:15.054371 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd8dc3-aead-4455-8986-e0495a1a15b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:15.071185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-b7dd8dc3-aead-4455-8986-e0495a1a15b8 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:19:15.072349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-b7dd8dc3-aead-4455-8986-e0495a1a15b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:19:15.143000 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 668/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:19:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:19:15.165924 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4356e65-4487-49f8-a4f0-198d33da1fd0 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:15.260395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4356e65-4487-49f8-a4f0-198d33da1fd0 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:15.293745 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:15.294517 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:15.294793 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:15.295060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:15.295698 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:15.297129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:15.308742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:15.309877 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:15.310943 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 630/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:19:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:15.316539 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd8dc3-aead-4455-8986-e0495a1a15b8 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:15.410278 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd8dc3-aead-4455-8986-e0495a1a15b8 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:15.441834 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:15.442478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:15.442756 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:15.443027 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:15.443724 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] RBAC: Adding query filter params (name=placement) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:15.444025 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:15.445453 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:15.450349 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 669/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:19:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:15.456596 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:15.553092 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7436749-06f4-495f-8408-929dcfbb8ed4 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:15.585142 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98dcfb-d2b4-46e9-bd36-bcf7995dc5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:15.585786 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98dcfb-d2b4-46e9-bd36-bcf7995dc5ac None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:15.586067 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98dcfb-d2b4-46e9-bd36-bcf7995dc5ac None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:15.586337 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98dcfb-d2b4-46e9-bd36-bcf7995dc5ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:15.587120 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98dcfb-d2b4-46e9-bd36-bcf7995dc5ac None admin] RBAC: Adding query filter params (interface=public, service_id=9c4564f04c904bb4bb923cce59ff8032, region_id=RegionOne) {{(pid=143116) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:15.587429 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98dcfb-d2b4-46e9-bd36-bcf7995dc5ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:15.588781 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98dcfb-d2b4-46e9-bd36-bcf7995dc5ac None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:15.593586 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 631/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:19:15 2022] GET /identity/v3/endpoints?service_id=9c4564f04c904bb4bb923cce59ff8032&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:15.599240 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:15.693230 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8653b7d-97ec-4b30-a9a5-6387e9661e64 None admin] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:15.724398 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9909ddd-9e49-4215-971b-1015b6cc9704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:15.725070 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9909ddd-9e49-4215-971b-1015b6cc9704 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:15.725353 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9909ddd-9e49-4215-971b-1015b6cc9704 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:15.725634 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9909ddd-9e49-4215-971b-1015b6cc9704 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:15.726310 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9909ddd-9e49-4215-971b-1015b6cc9704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:15.727657 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9909ddd-9e49-4215-971b-1015b6cc9704 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:15.734184 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 670/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:19:15 2022] GET /identity/v3/services => generated 2191 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:19:17.936333 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf69e89-c215-4744-8f36-25f0944d2436 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:17.936854 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf69e89-c215-4744-8f36-25f0944d2436 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:17.936968 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf69e89-c215-4744-8f36-25f0944d2436 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:17.938301 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 632/1302] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:19:17.944169 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e032fe7-2228-4555-b7b7-b887d2357b01 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:17.944582 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e032fe7-2228-4555-b7b7-b887d2357b01 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:17.944721 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e032fe7-2228-4555-b7b7-b887d2357b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:17.960798 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-2e032fe7-2228-4555-b7b7-b887d2357b01 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:19:17.962008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-2e032fe7-2228-4555-b7b7-b887d2357b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:19:18.024676 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 671/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:19:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:19:18.046595 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf69e89-c215-4744-8f36-25f0944d2436 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:18.142711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf69e89-c215-4744-8f36-25f0944d2436 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:18.174248 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:18.175041 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:18.175303 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:18.175572 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:18.176212 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:18.177589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:19:18.189559 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:19:18.190648 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application with Lock( May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self._enter() May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application return self.creator() May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application created_value = creator( May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: ERROR keystone.server.flask.application May 09 15:19:18.191792 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 633/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:19:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:19:18.196841 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e032fe7-2228-4555-b7b7-b887d2357b01 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:18.291055 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e032fe7-2228-4555-b7b7-b887d2357b01 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:18.323299 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d0989-c41f-40c6-9b0e-3619f3cc5587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:18.323949 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0989-c41f-40c6-9b0e-3619f3cc5587 None admin] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:18.324231 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0989-c41f-40c6-9b0e-3619f3cc5587 None admin] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:18.324505 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0989-c41f-40c6-9b0e-3619f3cc5587 None admin] PATH_INFO: `/v3/services` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:18.325231 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0989-c41f-40c6-9b0e-3619f3cc5587 None admin] RBAC: Adding query filter params (name=placement) {{(pid=143117) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:19:18.325534 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0989-c41f-40c6-9b0e-3619f3cc5587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:18.326889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0989-c41f-40c6-9b0e-3619f3cc5587 None admin] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:18.331806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 672/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:19:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:19:18.337485 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:19:18.432639 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2eb59d-5b88-4b42-a942-4bdeeb23f4c5 None admin] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:19:18.463452 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b1f25-d4fc-4a3e-afcd-286596a24019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de3600fcd96a4de8ae9f9640630b9125', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:19:18.464027 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1f25-d4fc-4a3e-afcd-286596a24019 None admin] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:19:18.464305 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1f25-d4fc-4a3e-afcd-286596a24019 None admin] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:19:18.464576 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b1f25-d4fc-4a3e-afcd-286596a24019 None admin] PATH_INFO: `/v3/endpoints` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:19:18.465329 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b1f25-d4fc-4a3e-afcd-286596a24019 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:19:18.466672 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b1f25-d4fc-4a3e-afcd-286596a24019 None admin] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:19:18.484681 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 634/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:19:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:20:23.026812 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b445-867c-4ba9-9f31-638212d3b260 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:23.027400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b445-867c-4ba9-9f31-638212d3b260 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:23.027400 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22b445-867c-4ba9-9f31-638212d3b260 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:23.028738 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 673/1307] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:23.034252 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ead25-e1ef-4794-acf8-5bb28d6c6016 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:23.035137 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ead25-e1ef-4794-acf8-5bb28d6c6016 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:23.035615 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ead25-e1ef-4794-acf8-5bb28d6c6016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:23.244703 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-b74ead25-e1ef-4794-acf8-5bb28d6c6016 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:23.253494 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-b74ead25-e1ef-4794-acf8-5bb28d6c6016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:23.464184 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 635/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:23.508003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767507ad-d625-47a5-9149-112a0fc19174 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:23.508309 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767507ad-d625-47a5-9149-112a0fc19174 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:23.508590 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767507ad-d625-47a5-9149-112a0fc19174 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:23.510884 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 674/1309] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 9 15:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:23.518098 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba5cc7-cb3c-4a13-8b36-c261e7e6193e None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:23.518395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba5cc7-cb3c-4a13-8b36-c261e7e6193e None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:23.518676 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba5cc7-cb3c-4a13-8b36-c261e7e6193e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:23.602874 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-bfba5cc7-cb3c-4a13-8b36-c261e7e6193e None None] MFA Rules not processed for user `9006561a3f5340c9b397c4892f7d1f34`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:23.604523 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-bfba5cc7-cb3c-4a13-8b36-c261e7e6193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:23.692277 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 636/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 9 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:23.701601 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767507ad-d625-47a5-9149-112a0fc19174 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:23.704122 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-767507ad-d625-47a5-9149-112a0fc19174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:23.821256 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767507ad-d625-47a5-9149-112a0fc19174 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:23.898608 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d808c8-c023-4ca7-bb3f-dbe0cddc144c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:23.899515 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d808c8-c023-4ca7-bb3f-dbe0cddc144c service neutron] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:23.899942 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d808c8-c023-4ca7-bb3f-dbe0cddc144c service neutron] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:23.900358 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d808c8-c023-4ca7-bb3f-dbe0cddc144c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:23.902208 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-10d808c8-c023-4ca7-bb3f-dbe0cddc144c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:24.028911 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d808c8-c023-4ca7-bb3f-dbe0cddc144c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:24.031087 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d808c8-c023-4ca7-bb3f-dbe0cddc144c service neutron] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:24.064904 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 675/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:26.633791 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30ecf-a007-4423-947e-27c830b73da5 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:26.634306 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30ecf-a007-4423-947e-27c830b73da5 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:26.634506 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a30ecf-a007-4423-947e-27c830b73da5 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:26.635997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 637/1312] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:26.642026 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057847-e0a8-4655-a088-59a1fa6a7f1d None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:26.642443 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057847-e0a8-4655-a088-59a1fa6a7f1d None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:26.642580 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f057847-e0a8-4655-a088-59a1fa6a7f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:26.724940 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-5f057847-e0a8-4655-a088-59a1fa6a7f1d None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:26.726249 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-5f057847-e0a8-4655-a088-59a1fa6a7f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:26.824356 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 676/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:26.862941 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a30ecf-a007-4423-947e-27c830b73da5 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:26.864925 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-e7a30ecf-a007-4423-947e-27c830b73da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:26.971381 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a30ecf-a007-4423-947e-27c830b73da5 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:27.062395 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b869fe-9224-4c5d-bf92-d9c7f0506fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:27.063057 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b869fe-9224-4c5d-bf92-d9c7f0506fbd service neutron] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:27.063336 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b869fe-9224-4c5d-bf92-d9c7f0506fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:27.063608 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b869fe-9224-4c5d-bf92-d9c7f0506fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:27.065343 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-06b869fe-9224-4c5d-bf92-d9c7f0506fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:27.186080 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b869fe-9224-4c5d-bf92-d9c7f0506fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:27.188089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b869fe-9224-4c5d-bf92-d9c7f0506fbd service neutron] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:27.208665 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 638/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:29.859570 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39f281-bd56-44a5-8391-d9addb9800d2 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:29.860185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39f281-bd56-44a5-8391-d9addb9800d2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:29.860185 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39f281-bd56-44a5-8391-d9addb9800d2 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:29.861598 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 677/1315] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:29.867793 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7945-feab-4a6f-bda4-16a026160cb5 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:29.868302 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7945-feab-4a6f-bda4-16a026160cb5 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:29.868429 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7945-feab-4a6f-bda4-16a026160cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:29.939469 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-7a1b7945-feab-4a6f-bda4-16a026160cb5 None None] MFA Rules not processed for user `cf20a77d54c04b289feef5d1d0955a73`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:29.940685 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-7a1b7945-feab-4a6f-bda4-16a026160cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:30.044551 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 639/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:30.084494 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39f281-bd56-44a5-8391-d9addb9800d2 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:30.086145 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-0b39f281-bd56-44a5-8391-d9addb9800d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:30.185601 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39f281-bd56-44a5-8391-d9addb9800d2 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:30.257124 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192f087-50ea-46fb-95b3-4e25a5a2449d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:30.257270 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192f087-50ea-46fb-95b3-4e25a5a2449d service neutron] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:30.257521 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192f087-50ea-46fb-95b3-4e25a5a2449d service neutron] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:30.257795 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192f087-50ea-46fb-95b3-4e25a5a2449d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:30.259478 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-5192f087-50ea-46fb-95b3-4e25a5a2449d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:30.375308 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192f087-50ea-46fb-95b3-4e25a5a2449d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:30.376754 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192f087-50ea-46fb-95b3-4e25a5a2449d service neutron] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:30.395116 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 678/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:33.215603 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2367-c621-4a72-8457-25558c4f650f None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:33.216083 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2367-c621-4a72-8457-25558c4f650f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:33.216187 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d2367-c621-4a72-8457-25558c4f650f None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:33.218159 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 640/1318] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:33.223336 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae0ba-b069-407b-b961-bada74d75fc1 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:33.223731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae0ba-b069-407b-b961-bada74d75fc1 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:33.223895 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae0ba-b069-407b-b961-bada74d75fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:33.286847 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-f3cae0ba-b069-407b-b961-bada74d75fc1 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:33.288024 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-f3cae0ba-b069-407b-b961-bada74d75fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:33.379694 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 679/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:33.418609 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d2367-c621-4a72-8457-25558c4f650f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:33.420060 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-da8d2367-c621-4a72-8457-25558c4f650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:33.518683 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d2367-c621-4a72-8457-25558c4f650f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:33.587995 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b127e6e-a266-4949-9426-9ee09d245267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:33.588630 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127e6e-a266-4949-9426-9ee09d245267 service neutron] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:33.588941 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127e6e-a266-4949-9426-9ee09d245267 service neutron] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:33.589224 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127e6e-a266-4949-9426-9ee09d245267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:33.590863 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-1b127e6e-a266-4949-9426-9ee09d245267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:33.695128 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b127e6e-a266-4949-9426-9ee09d245267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:33.696486 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b127e6e-a266-4949-9426-9ee09d245267 service neutron] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:33.713744 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 641/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:36.605469 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43807f62-3f94-490a-b78e-f418c4ce5b49 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:36.606008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43807f62-3f94-490a-b78e-f418c4ce5b49 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:36.606008 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43807f62-3f94-490a-b78e-f418c4ce5b49 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:36.607314 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 680/1321] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:36.612694 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621e4c-57ae-4534-8f39-fee4dee3ade6 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:36.613129 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621e4c-57ae-4534-8f39-fee4dee3ade6 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:36.613296 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43621e4c-57ae-4534-8f39-fee4dee3ade6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:36.676490 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-43621e4c-57ae-4534-8f39-fee4dee3ade6 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:36.677711 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-43621e4c-57ae-4534-8f39-fee4dee3ade6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:36.772356 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 642/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:36.808056 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43807f62-3f94-490a-b78e-f418c4ce5b49 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:36.809731 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-43807f62-3f94-490a-b78e-f418c4ce5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:36.910081 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43807f62-3f94-490a-b78e-f418c4ce5b49 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:36.981173 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdece30-de89-41bd-a78c-5be20c848ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:36.981819 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdece30-de89-41bd-a78c-5be20c848ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:36.982100 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdece30-de89-41bd-a78c-5be20c848ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:36.982374 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdece30-de89-41bd-a78c-5be20c848ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:36.984538 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-cfdece30-de89-41bd-a78c-5be20c848ac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:37.096689 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdece30-de89-41bd-a78c-5be20c848ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:37.098161 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdece30-de89-41bd-a78c-5be20c848ac8 service neutron] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:37.116806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 681/1323] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:39.427227 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6779-ac8c-4af1-a84b-d6a35eba10a7 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:39.427720 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6779-ac8c-4af1-a84b-d6a35eba10a7 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:39.427720 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad6779-ac8c-4af1-a84b-d6a35eba10a7 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:39.429283 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 643/1324] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:39.434470 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dafe9-10c9-4ed2-820d-7bc42fdb2d75 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:39.434899 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dafe9-10c9-4ed2-820d-7bc42fdb2d75 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:39.435070 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dafe9-10c9-4ed2-820d-7bc42fdb2d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:39.519877 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-ab0dafe9-10c9-4ed2-820d-7bc42fdb2d75 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:39.521134 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-ab0dafe9-10c9-4ed2-820d-7bc42fdb2d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:39.617911 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 682/1325] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:39.653446 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad6779-ac8c-4af1-a84b-d6a35eba10a7 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:39.654946 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-ecad6779-ac8c-4af1-a84b-d6a35eba10a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:39.755505 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad6779-ac8c-4af1-a84b-d6a35eba10a7 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:39.826236 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af49f9a-7c0d-43b2-8e59-ffcc09f049e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:39.826884 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49f9a-7c0d-43b2-8e59-ffcc09f049e3 service neutron] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:39.827165 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49f9a-7c0d-43b2-8e59-ffcc09f049e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:39.827441 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49f9a-7c0d-43b2-8e59-ffcc09f049e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:39.829616 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-8af49f9a-7c0d-43b2-8e59-ffcc09f049e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:39.941310 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af49f9a-7c0d-43b2-8e59-ffcc09f049e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:39.942740 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af49f9a-7c0d-43b2-8e59-ffcc09f049e3 service neutron] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:39.962088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 644/1326] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:42.223666 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a827f42-3864-48d6-bbce-f2825d8be313 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:42.224188 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a827f42-3864-48d6-bbce-f2825d8be313 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:42.224311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a827f42-3864-48d6-bbce-f2825d8be313 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:42.225632 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 683/1327] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:42.231088 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e207-baf5-419e-88d3-eea316a552b5 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:42.231524 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e207-baf5-419e-88d3-eea316a552b5 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:42.231644 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e207-baf5-419e-88d3-eea316a552b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:42.294467 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-0328e207-baf5-419e-88d3-eea316a552b5 None None] MFA Rules not processed for user `cf20a77d54c04b289feef5d1d0955a73`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:42.295651 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-0328e207-baf5-419e-88d3-eea316a552b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:42.387474 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 645/1328] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:42.423856 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a827f42-3864-48d6-bbce-f2825d8be313 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:42.425395 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-1a827f42-3864-48d6-bbce-f2825d8be313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:42.523023 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a827f42-3864-48d6-bbce-f2825d8be313 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:42.598155 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca4bd6-cf7b-498f-a065-5bdabb9c8776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:42.598794 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4bd6-cf7b-498f-a065-5bdabb9c8776 service neutron] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:42.599067 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4bd6-cf7b-498f-a065-5bdabb9c8776 service neutron] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:42.599335 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca4bd6-cf7b-498f-a065-5bdabb9c8776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:42.600950 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-4fca4bd6-cf7b-498f-a065-5bdabb9c8776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:42.702736 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca4bd6-cf7b-498f-a065-5bdabb9c8776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:42.704157 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca4bd6-cf7b-498f-a065-5bdabb9c8776 service neutron] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:42.721203 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 684/1329] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:45.186169 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e4b1b-50d5-4b76-87f4-7f944cfc87fb None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:45.186580 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e4b1b-50d5-4b76-87f4-7f944cfc87fb None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:45.186725 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e4b1b-50d5-4b76-87f4-7f944cfc87fb None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:45.188730 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 646/1330] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:45.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc607-8149-4d7f-86cc-ff4ab17c659e None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:45.194363 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc607-8149-4d7f-86cc-ff4ab17c659e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:45.194494 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dc607-8149-4d7f-86cc-ff4ab17c659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:45.258311 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-559dc607-8149-4d7f-86cc-ff4ab17c659e None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:45.259530 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-559dc607-8149-4d7f-86cc-ff4ab17c659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:45.351632 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 685/1331] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:45.387796 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e4b1b-50d5-4b76-87f4-7f944cfc87fb None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:45.389337 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-235e4b1b-50d5-4b76-87f4-7f944cfc87fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:45.490910 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e4b1b-50d5-4b76-87f4-7f944cfc87fb None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:45.560691 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8543a4a-98b5-4199-a95c-5f79af6af093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:45.561408 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543a4a-98b5-4199-a95c-5f79af6af093 service neutron] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:45.562200 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543a4a-98b5-4199-a95c-5f79af6af093 service neutron] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:45.562500 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8543a4a-98b5-4199-a95c-5f79af6af093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:45.564161 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-b8543a4a-98b5-4199-a95c-5f79af6af093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:45.668062 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8543a4a-98b5-4199-a95c-5f79af6af093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:45.669526 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8543a4a-98b5-4199-a95c-5f79af6af093 service neutron] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:45.687960 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 647/1332] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:47.950961 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58507b7d-0bb7-4c4d-bb96-c937c4286af2 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:47.951513 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58507b7d-0bb7-4c4d-bb96-c937c4286af2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:47.951623 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58507b7d-0bb7-4c4d-bb96-c937c4286af2 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:47.953272 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 686/1333] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:47.958671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa8f7-4900-4e81-bad3-90b9f38daa34 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:47.959059 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa8f7-4900-4e81-bad3-90b9f38daa34 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:47.959230 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa8f7-4900-4e81-bad3-90b9f38daa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:48.022416 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-9bffa8f7-4900-4e81-bad3-90b9f38daa34 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:48.023618 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-9bffa8f7-4900-4e81-bad3-90b9f38daa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:48.115529 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 648/1334] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:48.154780 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58507b7d-0bb7-4c4d-bb96-c937c4286af2 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:48.156946 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-58507b7d-0bb7-4c4d-bb96-c937c4286af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:48.255614 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58507b7d-0bb7-4c4d-bb96-c937c4286af2 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:48.326202 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048bdfd-7037-4223-a1d8-8184ffcfefc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:48.326848 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048bdfd-7037-4223-a1d8-8184ffcfefc8 service neutron] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:48.327130 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048bdfd-7037-4223-a1d8-8184ffcfefc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:48.328003 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048bdfd-7037-4223-a1d8-8184ffcfefc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:48.329670 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-6048bdfd-7037-4223-a1d8-8184ffcfefc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:48.431610 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048bdfd-7037-4223-a1d8-8184ffcfefc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:48.433032 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048bdfd-7037-4223-a1d8-8184ffcfefc8 service neutron] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:48.450619 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 687/1335] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:51.014387 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479f7d6-055a-410b-b032-f1747872a30f None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:51.014922 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479f7d6-055a-410b-b032-f1747872a30f None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:51.015028 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479f7d6-055a-410b-b032-f1747872a30f None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:51.016838 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 649/1336] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:51.021983 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15272b-4d0f-4432-847e-82d1deadd9a4 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:51.022366 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15272b-4d0f-4432-847e-82d1deadd9a4 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:51.022545 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15272b-4d0f-4432-847e-82d1deadd9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:51.084310 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-9a15272b-4d0f-4432-847e-82d1deadd9a4 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:51.085546 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-9a15272b-4d0f-4432-847e-82d1deadd9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:51.184526 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 688/1337] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:51.220603 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479f7d6-055a-410b-b032-f1747872a30f None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:51.222183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-9479f7d6-055a-410b-b032-f1747872a30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:51.324019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479f7d6-055a-410b-b032-f1747872a30f None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:51.394556 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebadab-6256-45d4-ad57-b1b03d8f7c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:51.395196 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebadab-6256-45d4-ad57-b1b03d8f7c74 service neutron] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:51.395474 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebadab-6256-45d4-ad57-b1b03d8f7c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:51.395746 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebadab-6256-45d4-ad57-b1b03d8f7c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:51.397442 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-e6ebadab-6256-45d4-ad57-b1b03d8f7c74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:51.501508 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebadab-6256-45d4-ad57-b1b03d8f7c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:51.502935 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebadab-6256-45d4-ad57-b1b03d8f7c74 service neutron] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:51.520733 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 650/1338] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:55.718461 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63dee-d7d7-4c3c-9888-991a671a891e None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:55.719004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63dee-d7d7-4c3c-9888-991a671a891e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:55.719004 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd63dee-d7d7-4c3c-9888-991a671a891e None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:55.720989 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 689/1339] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:55.726249 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e9781-3a66-44c1-91ce-d1bc95efb214 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:55.726671 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e9781-3a66-44c1-91ce-d1bc95efb214 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:55.726927 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e9781-3a66-44c1-91ce-d1bc95efb214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:55.795959 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-a46e9781-3a66-44c1-91ce-d1bc95efb214 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:55.797089 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-a46e9781-3a66-44c1-91ce-d1bc95efb214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:55.897185 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 651/1340] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:55.934739 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd63dee-d7d7-4c3c-9888-991a671a891e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:55.936838 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-ccd63dee-d7d7-4c3c-9888-991a671a891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:56.031504 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd63dee-d7d7-4c3c-9888-991a671a891e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:56.035046 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7d114-cb83-4e72-bd99-587618ba7ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:56.035696 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d114-cb83-4e72-bd99-587618ba7ccb service neutron] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:56.035971 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d114-cb83-4e72-bd99-587618ba7ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:56.036242 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7d114-cb83-4e72-bd99-587618ba7ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:56.045420 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7d114-cb83-4e72-bd99-587618ba7ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:56.046854 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7d114-cb83-4e72-bd99-587618ba7ccb service neutron] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:56.056049 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 690/1341] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:20:58.892155 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937ce0d-8c75-4e88-89ff-78259aa258f0 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:58.892868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937ce0d-8c75-4e88-89ff-78259aa258f0 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:58.892868 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937ce0d-8c75-4e88-89ff-78259aa258f0 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:58.894049 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 652/1342] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:58.899232 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ce541-2950-48b5-86f0-49dec6e605f2 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:58.899639 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ce541-2950-48b5-86f0-49dec6e605f2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:58.899794 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ce541-2950-48b5-86f0-49dec6e605f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:58.926411 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-829ce541-2950-48b5-86f0-49dec6e605f2 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:58.927549 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-829ce541-2950-48b5-86f0-49dec6e605f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:59.007454 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 691/1343] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:20:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:20:59.060280 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5b6f3-3d46-4c2a-9de0-de2625fcef00 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:59.060570 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5b6f3-3d46-4c2a-9de0-de2625fcef00 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:59.060853 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5b6f3-3d46-4c2a-9de0-de2625fcef00 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:59.062325 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 653/1344] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 9 15:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:20:59.069114 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760afa-f9c2-4a85-bbe5-58fb729069b3 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:59.069413 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760afa-f9c2-4a85-bbe5-58fb729069b3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:59.069692 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760afa-f9c2-4a85-bbe5-58fb729069b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:59.124766 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-63760afa-f9c2-4a85-bbe5-58fb729069b3 None None] MFA Rules not processed for user `9006561a3f5340c9b397c4892f7d1f34`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:20:59.126528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-63760afa-f9c2-4a85-bbe5-58fb729069b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:20:59.133015 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 692/1345] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 9 15:20:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:20:59.142349 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5b6f3-3d46-4c2a-9de0-de2625fcef00 None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:20:59.152870 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5b6f3-3d46-4c2a-9de0-de2625fcef00 None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:20:59.156505 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9c5cd-094d-401d-b824-e511cab3579c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:20:59.157300 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9c5cd-094d-401d-b824-e511cab3579c service neutron] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:20:59.157589 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9c5cd-094d-401d-b824-e511cab3579c service neutron] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:20:59.157869 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9c5cd-094d-401d-b824-e511cab3579c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:20:59.167566 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9c5cd-094d-401d-b824-e511cab3579c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:20:59.168997 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e9c5cd-094d-401d-b824-e511cab3579c service neutron] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:20:59.179514 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 654/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:20:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:21:04.255068 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94709c17-902b-489e-9a0c-2c1f992185e3 None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:04.255500 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94709c17-902b-489e-9a0c-2c1f992185e3 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:04.255680 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94709c17-902b-489e-9a0c-2c1f992185e3 None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:04.257018 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 693/1347] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:04.262575 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f0540-0536-4428-9818-0145ba708af8 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:04.263019 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f0540-0536-4428-9818-0145ba708af8 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:04.263135 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f0540-0536-4428-9818-0145ba708af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:04.280375 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-d86f0540-0536-4428-9818-0145ba708af8 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:04.282114 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-d86f0540-0536-4428-9818-0145ba708af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:04.289897 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 655/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:04.327747 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94709c17-902b-489e-9a0c-2c1f992185e3 None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:21:04.338222 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94709c17-902b-489e-9a0c-2c1f992185e3 None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:21:04.341792 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc28c5-f1fb-43aa-9717-d355c7e4dd5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9006561a3f5340c9b397c4892f7d1f34', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:21:04.342491 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc28c5-f1fb-43aa-9717-d355c7e4dd5d service neutron] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:04.342844 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc28c5-f1fb-43aa-9717-d355c7e4dd5d service neutron] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:04.343119 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc28c5-f1fb-43aa-9717-d355c7e4dd5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:04.352315 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc28c5-f1fb-43aa-9717-d355c7e4dd5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:21:04.353769 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc28c5-f1fb-43aa-9717-d355c7e4dd5d service neutron] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:21:04.363985 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 694/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:21:09.715522 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f9ff-97ae-4b47-bb5f-bb3424f40e9e None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:09.716009 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f9ff-97ae-4b47-bb5f-bb3424f40e9e None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:09.716825 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125f9ff-97ae-4b47-bb5f-bb3424f40e9e None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:09.718664 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 656/1350] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:09.724341 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e658b0-9cb2-413d-b7f9-400fe3a07fd9 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:09.724775 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e658b0-9cb2-413d-b7f9-400fe3a07fd9 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:09.724896 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e658b0-9cb2-413d-b7f9-400fe3a07fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:09.774725 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-e2e658b0-9cb2-413d-b7f9-400fe3a07fd9 None None] MFA Rules not processed for user `c22d09c751714c30b4d1bb63e1323e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:09.776274 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-e2e658b0-9cb2-413d-b7f9-400fe3a07fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:09.852806 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 695/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:21:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:14.261617 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80032c24-7014-498e-9a2c-ac727b4aefbc None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:14.262461 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80032c24-7014-498e-9a2c-ac727b4aefbc None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:14.262751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80032c24-7014-498e-9a2c-ac727b4aefbc None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:14.264323 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 657/1352] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:14.269889 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3d03b-f00e-4c00-a3cf-b9b11be2bb47 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:14.270348 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3d03b-f00e-4c00-a3cf-b9b11be2bb47 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:14.270634 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3d03b-f00e-4c00-a3cf-b9b11be2bb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:14.290368 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-82b3d03b-f00e-4c00-a3cf-b9b11be2bb47 None None] MFA Rules not processed for user `c22d09c751714c30b4d1bb63e1323e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:14.291748 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-82b3d03b-f00e-4c00-a3cf-b9b11be2bb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:14.300715 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 696/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:15.805278 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d7f4c-fa6d-4384-aea2-43f7723ad343 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:15.805774 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d7f4c-fa6d-4384-aea2-43f7723ad343 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:15.805892 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d7f4c-fa6d-4384-aea2-43f7723ad343 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:15.807213 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 658/1354] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:15.812995 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308645d-bafa-4be9-81ef-2e178b7943b2 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:15.813425 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308645d-bafa-4be9-81ef-2e178b7943b2 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:15.813717 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308645d-bafa-4be9-81ef-2e178b7943b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:15.832407 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-0308645d-bafa-4be9-81ef-2e178b7943b2 None None] MFA Rules not processed for user `c22d09c751714c30b4d1bb63e1323e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:15.833743 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-0308645d-bafa-4be9-81ef-2e178b7943b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:15.842256 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 697/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:21:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:21:19.615254 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ee40-6bd0-4fe2-9100-db3004ba74d1 None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:19.615751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ee40-6bd0-4fe2-9100-db3004ba74d1 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:19.615751 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ee40-6bd0-4fe2-9100-db3004ba74d1 None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:19.617244 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 659/1356] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:19.622253 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c6886-cd6a-4c01-89d0-825caabf4ba1 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:19.622643 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c6886-cd6a-4c01-89d0-825caabf4ba1 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:19.622817 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c6886-cd6a-4c01-89d0-825caabf4ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:19.640888 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-d07c6886-cd6a-4c01-89d0-825caabf4ba1 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:19.642126 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-d07c6886-cd6a-4c01-89d0-825caabf4ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:19.650231 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 698/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:19.697742 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0f19-bac8-48c2-9184-913c45c4e3dc None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:19.698032 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0f19-bac8-48c2-9184-913c45c4e3dc None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:19.698318 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0f19-bac8-48c2-9184-913c45c4e3dc None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:19.699963 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 660/1358] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 9 15:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:19.706129 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c84855-80ae-4925-babe-9892be3d8dee None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:19.706425 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c84855-80ae-4925-babe-9892be3d8dee None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:19.706710 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c84855-80ae-4925-babe-9892be3d8dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:19.761344 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-70c84855-80ae-4925-babe-9892be3d8dee None None] MFA Rules not processed for user `f3a73bc43c23400d8ee7d4e880aa7242`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:19.763420 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-70c84855-80ae-4925-babe-9892be3d8dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:19.865853 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 699/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 9 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:21:19.867130 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0f19-bac8-48c2-9184-913c45c4e3dc None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:21:19.885139 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0f19-bac8-48c2-9184-913c45c4e3dc None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:21:19.892380 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab05e3c-f9bb-4d00-8922-1316c12fb0e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a73bc43c23400d8ee7d4e880aa7242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:21:19.893324 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab05e3c-f9bb-4d00-8922-1316c12fb0e5 service nova] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:19.893891 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab05e3c-f9bb-4d00-8922-1316c12fb0e5 service nova] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:19.894365 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab05e3c-f9bb-4d00-8922-1316c12fb0e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:19.903833 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab05e3c-f9bb-4d00-8922-1316c12fb0e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:21:19.905837 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab05e3c-f9bb-4d00-8922-1316c12fb0e5 service nova] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:21:19.916813 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 661/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:21:22.390551 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b4fb5-a101-4553-af7c-2c74b85ca85e None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:22.391028 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b4fb5-a101-4553-af7c-2c74b85ca85e None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:22.391133 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b4fb5-a101-4553-af7c-2c74b85ca85e None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:22.392529 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 700/1361] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:22.398058 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bf6d-5a40-43b9-a53f-c46144daef34 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:22.398447 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bf6d-5a40-43b9-a53f-c46144daef34 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:22.398695 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bf6d-5a40-43b9-a53f-c46144daef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:22.416540 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-d9e2bf6d-5a40-43b9-a53f-c46144daef34 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:22.417852 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-d9e2bf6d-5a40-43b9-a53f-c46144daef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:22.426660 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 662/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:22.470862 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b4fb5-a101-4553-af7c-2c74b85ca85e None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:21:22.486230 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b4fb5-a101-4553-af7c-2c74b85ca85e None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:21:22.491883 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2647ab3-6133-466d-be9b-0c3f792c42e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a73bc43c23400d8ee7d4e880aa7242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:21:22.492541 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2647ab3-6133-466d-be9b-0c3f792c42e7 service nova] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:22.492829 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2647ab3-6133-466d-be9b-0c3f792c42e7 service nova] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:22.493144 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2647ab3-6133-466d-be9b-0c3f792c42e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:22.502652 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2647ab3-6133-466d-be9b-0c3f792c42e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:21:22.504222 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2647ab3-6133-466d-be9b-0c3f792c42e7 service nova] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:21:22.515908 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 701/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:21:25.075043 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad349-f25d-4eea-98bd-3c240426006c None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:25.076005 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad349-f25d-4eea-98bd-3c240426006c None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:25.076303 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad349-f25d-4eea-98bd-3c240426006c None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:25.077965 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 663/1364] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:25.083251 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5421690-3e82-4f8e-9dea-669bd8bf3901 None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:25.083674 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5421690-3e82-4f8e-9dea-669bd8bf3901 None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:25.083807 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5421690-3e82-4f8e-9dea-669bd8bf3901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:25.102266 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-e5421690-3e82-4f8e-9dea-669bd8bf3901 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:25.103676 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-e5421690-3e82-4f8e-9dea-669bd8bf3901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:25.112627 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 702/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:25.157149 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de8c98-5148-4482-83fe-5a7cabd3acbc None None] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:25.157718 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de8c98-5148-4482-83fe-5a7cabd3acbc None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:25.158007 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de8c98-5148-4482-83fe-5a7cabd3acbc None None] PATH_INFO: `/` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:25.159440 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 664/1366] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 9 15:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:25.164855 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004f57f-b9af-4bd0-91d6-78feaa16878f None None] REQUEST_METHOD: `POST` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:25.165195 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004f57f-b9af-4bd0-91d6-78feaa16878f None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:25.165479 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004f57f-b9af-4bd0-91d6-78feaa16878f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:25.184709 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.auth.core [None req-8004f57f-b9af-4bd0-91d6-78feaa16878f None None] MFA Rules not processed for user `f3a73bc43c23400d8ee7d4e880aa7242`. Rule list: `[]` (Enabled: `True`). {{(pid=143117) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:25.185941 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.fernet_utils [None req-8004f57f-b9af-4bd0-91d6-78feaa16878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143117) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:25.195416 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 703/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 9 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:25.203202 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de8c98-5148-4482-83fe-5a7cabd3acbc None None] Authenticating user token {{(pid=143116) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:21:25.217476 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de8c98-5148-4482-83fe-5a7cabd3acbc None None] Validating token access rules against request {{(pid=143116) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:21:25.223539 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fc198-5d3c-4cdd-845a-727deb1c0af5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a73bc43c23400d8ee7d4e880aa7242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143116) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:21:25.224186 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc198-5d3c-4cdd-845a-727deb1c0af5 service nova] REQUEST_METHOD: `GET` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:25.224474 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc198-5d3c-4cdd-845a-727deb1c0af5 service nova] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:25.224752 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc198-5d3c-4cdd-845a-727deb1c0af5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:25.232887 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fc198-5d3c-4cdd-845a-727deb1c0af5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:21:25.235080 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fc198-5d3c-4cdd-845a-727deb1c0af5 service nova] RBAC: Authorization granted {{(pid=143116) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:21:25.245183 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 665/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:21:26.059214 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fffcb-fd35-4b57-8884-de76ccf6e10b None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:26.059528 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fffcb-fd35-4b57-8884-de76ccf6e10b None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:26.059781 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fffcb-fd35-4b57-8884-de76ccf6e10b None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:26.061355 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 704/1369] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:26.067111 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e52e46-5211-42d0-be20-44e0e5ecc370 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:26.067412 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e52e46-5211-42d0-be20-44e0e5ecc370 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:26.067692 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e52e46-5211-42d0-be20-44e0e5ecc370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:26.085987 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-06e52e46-5211-42d0-be20-44e0e5ecc370 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:26.087194 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-06e52e46-5211-42d0-be20-44e0e5ecc370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:26.097171 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 666/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:27.609953 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238352-9230-4685-8232-174c0edd477a None None] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:27.610451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238352-9230-4685-8232-174c0edd477a None None] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:27.610451 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238352-9230-4685-8232-174c0edd477a None None] PATH_INFO: `/` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:27.611808 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 705/1371] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:21:27.617423 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7229b-8705-4aaa-9faa-258fcc75c126 None None] REQUEST_METHOD: `POST` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:27.617828 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7229b-8705-4aaa-9faa-258fcc75c126 None None] SCRIPT_NAME: `/identity` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:27.618106 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7229b-8705-4aaa-9faa-258fcc75c126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=143116) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:27.635892 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.auth.core [None req-0cc7229b-8705-4aaa-9faa-258fcc75c126 None None] MFA Rules not processed for user `de3600fcd96a4de8ae9f9640630b9125`. Rule list: `[]` (Enabled: `True`). {{(pid=143116) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:21:27.637095 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: DEBUG keystone.common.fernet_utils [None req-0cc7229b-8705-4aaa-9faa-258fcc75c126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=143116) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:21:27.645374 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: [pid: 143116|app: 0|req: 667/1372] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:21:27.690133 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d238352-9230-4685-8232-174c0edd477a None None] Authenticating user token {{(pid=143117) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:21:27.704701 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d238352-9230-4685-8232-174c0edd477a None None] Validating token access rules against request {{(pid=143117) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:21:27.710108 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddde60f-8dea-4ee5-8dfa-54e010c5b0bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3a73bc43c23400d8ee7d4e880aa7242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5861448983e849eaa5ee1cc05ffc2694', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=143117) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:21:27.710740 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde60f-8dea-4ee5-8dfa-54e010c5b0bc service nova] REQUEST_METHOD: `GET` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:21:27.711021 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde60f-8dea-4ee5-8dfa-54e010c5b0bc service nova] SCRIPT_NAME: `/identity` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:21:27.711295 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde60f-8dea-4ee5-8dfa-54e010c5b0bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=143117) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:21:27.720053 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddde60f-8dea-4ee5-8dfa-54e010c5b0bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:21:27.721814 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddde60f-8dea-4ee5-8dfa-54e010c5b0bc service nova] RBAC: Authorization granted {{(pid=143117) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:21:27.731250 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: [pid: 143117|app: 0|req: 706/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 15:21:49.839774 dv-cinder-840678-3-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 09 15:21:49.840201 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: Mon May 9 15:21:49 2022 - graceful shutdown triggered... May 09 15:21:49.840627 dv-cinder-840678-3-iscsi devstack@keystone.service[143116]: Gracefully killing worker 1 (pid: 143116)... May 09 15:21:49.841128 dv-cinder-840678-3-iscsi devstack@keystone.service[143117]: Gracefully killing worker 2 (pid: 143117)... May 09 15:21:50.841300 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: worker 1 buried after 1 seconds May 09 15:21:50.841825 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: worker 2 buried after 1 seconds May 09 15:21:50.841825 dv-cinder-840678-3-iscsi devstack@keystone.service[143100]: goodbye to uWSGI. May 09 15:21:50.842836 dv-cinder-840678-3-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 09 15:21:50.843374 dv-cinder-840678-3-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 09 15:27:13.142333 dv-cinder-840678-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 09 15:27:13.151369 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 9 15:27:13 2022] *** May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: nodename: dv-cinder-840678-3-iscsi May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: machine: x86_64 May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: clock source: unix May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: pcre jit disabled May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: detected number of CPU cores: 4 May 09 15:27:13.154211 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: current working directory: / May 09 15:27:13.154808 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: detected binary path: /usr/bin/uwsgi-core May 09 15:27:13.154808 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: your processes number limit is 47680 May 09 15:27:13.154808 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: your memory page size is 4096 bytes May 09 15:27:13.154808 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: detected max file descriptor number: 2048 May 09 15:27:13.154808 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: lock engine: pthread robust mutexes May 09 15:27:13.154808 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: thunder lock: enabled May 09 15:27:13.154808 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 09 15:27:13.155218 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 09 15:27:13.202707 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: Python main interpreter initialized at 0x563fb82d99c0 May 09 15:27:13.202707 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: python threads support enabled May 09 15:27:13.202707 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: your server socket listen backlog is limited to 100 connections May 09 15:27:13.202707 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: your mercy for graceful operations on workers is 90 seconds May 09 15:27:13.203156 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: mapped 403077 bytes (393 KB) for 2 cores May 09 15:27:13.203156 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: *** Operational MODE: preforking *** May 09 15:27:13.203403 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: *** uWSGI is running in multiple interpreter mode *** May 09 15:27:13.203403 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: spawned uWSGI master process (pid: 206268) May 09 15:27:13.203615 dv-cinder-840678-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 09 15:27:13.206523 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: spawned uWSGI worker 1 (pid: 206280, cores: 1) May 09 15:27:13.206523 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: spawned uWSGI worker 2 (pid: 206281, cores: 1) May 09 15:27:13.206523 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 09 15:27:18.996526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:27:18.996526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: from cryptography.utils import int_from_bytes May 09 15:27:18.997432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:27:18.997432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: from cryptography.utils import int_from_bytes May 09 15:27:18.997917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:27:18.997917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: from cryptography.utils import int_from_bytes May 09 15:27:18.998670 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:27:18.998670 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: from cryptography.utils import int_from_bytes May 09 15:27:19.047410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 15:27:19.047599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 15:27:19.047703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 15:27:19.047808 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] command line args: [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 15:27:19.047904 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 15:27:19.047990 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] command line args: [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 15:27:19.048074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 15:27:19.048208 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ================================================================================ {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 15:27:19.048303 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 15:27:19.048390 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ================================================================================ {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 15:27:19.048475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] admin_token = **** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.048639 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] admin_token = **** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.048733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] config_dir = [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.048870 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.048964 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] config_dir = [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049089 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] config_source = [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] debug = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049362 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] config_source = [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] debug = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049538 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049837 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.049929 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] insecure_debug = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] insecure_debug = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050335 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050546 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] list_limit = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050755 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] list_limit = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050830 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] log_config_append = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.050902 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] log_config_append = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051026 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051121 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051202 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] log_dir = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] log_dir = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051351 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] log_file = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051419 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] log_file = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051581 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.051944 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052505 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052817 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052909 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.052996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.053161 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.053261 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.053398 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.053567 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.053665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.053750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.053858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.053951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.054086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.054173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.054326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.054489 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.054580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] notification_format = cadf {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.054664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] notification_format = cadf {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.054834 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.054934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.055321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.055403 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.055496 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] publish_errors = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.055589 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] publish_errors = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.055725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.055818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.055957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056047 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056212 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056370 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056462 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056839 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] standard_threads = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.056924 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] standard_threads = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] strict_password_check = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] strict_password_check = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057337 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] use_eventlog = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057597 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] use_eventlog = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] use_journal = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057772 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] use_journal = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] use_json = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.057934 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] use_json = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.058018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] use_stderr = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.058102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] use_stderr = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.058232 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] use_syslog = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.058313 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] use_syslog = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.058397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] watch_log_file = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.058486 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] watch_log_file = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:27:19.058627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.058726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.058866 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.058956 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.059127 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.059233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.059366 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.059472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.059615 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.059707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.059855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.059946 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.060105 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.060203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.060339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.060430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.060586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.060677 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.060833 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.060931 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.061103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.061193 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.061341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.061427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.061592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.061690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.061828 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.061923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.062077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.062167 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.062323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.062418 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.062561 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.062652 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.062815 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.062916 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.063070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.063168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.063315 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.063407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.063549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.063639 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.063801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.063982 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.064075 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.064159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.064302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.064401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.064554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.064652 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.064784 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.064869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065367 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.065996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.066088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.066226 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.066317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.066455 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.066547 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.066702 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.066793 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.066948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.067044 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.067187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.067278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.067459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.067558 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.067694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.067786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.067930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068023 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068256 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068393 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068483 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068623 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.068942 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.069100 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.069201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.069339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.069426 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.069588 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.069675 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.069805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.069899 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070041 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070270 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070502 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070734 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.070978 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.071069 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.071206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.071297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.071440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.071525 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.071677 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.071768 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.071911 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072002 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072147 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072238 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.072948 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.073152 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.073237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.073344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.073436 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.073574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.073670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.073830 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.073926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.074063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.074148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.074294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.074385 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.074530 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.074615 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.074764 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.074850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075356 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075604 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075756 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075846 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.075990 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.076081 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.076223 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.076313 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.076455 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.076546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.076808 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.enabled = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.076901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.enabled = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.077063 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.077161 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.077303 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.077401 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.077549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.077640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.077789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.077880 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.078028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.078126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.078263 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.078501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.078592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.078736 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.078830 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079092 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079721 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079812 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.079965 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.080057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.080207 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.080317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.080445 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.080531 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.080683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.080773 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.080915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081268 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081491 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081646 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081744 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.081970 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.082122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.082209 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.082360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.082458 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.082610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.082700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.082848 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.082940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.083629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.083726 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.083892 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.083983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.084133 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.084224 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.084366 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] auth.external = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.084479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] auth.external = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.084602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] auth.mapped = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.084727 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] auth.mapped = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.084901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085158 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085249 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] auth.password = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] auth.password = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085626 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] auth.token = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085717 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] auth.token = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.085958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.086107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] catalog.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.086198 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] catalog.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.086344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.086433 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.086584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.086675 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.086831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.086917 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.087078 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.087170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.087311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.087403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.087537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] credential.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.087617 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] credential.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.087786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] credential.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.087877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] credential.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088271 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088352 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088515 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088607 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088844 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.088990 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.089111 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.089277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.089370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.089537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.089653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.089770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.089857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.090022 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.090106 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.090306 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.090400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.090557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.090644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.090801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.090890 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.091041 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.091134 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.091298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.091391 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.091536 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] federation.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.091630 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] federation.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.091773 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.091865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] federation.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] federation.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092355 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092831 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.092974 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.093095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.093258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.093374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.093494 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.093615 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.093755 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.093997 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.094088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.094246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.094390 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.094493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.094741 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.094871 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.094998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.095195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.095284 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.095361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.095581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.095677 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.095748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.095827 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.096077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.096172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.096243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.096315 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.096557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.096655 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.096727 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.096794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.097072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.097176 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.097247 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.097324 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.097994 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.098068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.098143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.098216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.098296 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.098365 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.098430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.098501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.098573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.099266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.099357 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.099432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.099505 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.099584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.099712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.099820 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.099954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100050 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100899 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.100984 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.101157 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.101269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.101401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.101555 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.101651 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.101739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.101883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.101979 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.102114 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.102206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.102361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.102452 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.102606 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.102701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.102840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.102925 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.103157 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.103254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.103341 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.103434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.103550 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.103643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.103787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.103884 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.password = **** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.password = **** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104490 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104828 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.104967 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.105125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.105221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.105307 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.105469 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.105573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.105717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.105809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.105946 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106032 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106270 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106422 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106514 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106755 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106902 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.106999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.107136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.107221 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.107371 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.107725 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.107857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.107955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.108098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.108232 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.108363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.108452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.108594 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.108695 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.108841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.108933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.109111 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.109211 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.109359 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.109454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.109593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.109678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.109823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.109915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.110058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.110176 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.110295 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.110421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.110534 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.110663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.110750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.110894 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111393 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111486 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111734 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111895 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.111992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.112145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.112237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.112377 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.112464 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.112627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.112720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.112866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.112952 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.113127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.113248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.113369 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.113463 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.113619 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.113707 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.113870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.113956 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.114111 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.114203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.114375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.114462 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] policy.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.114623 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.114711 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] policy.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.114854 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.114939 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.115091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.115185 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.115326 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.115417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] receipt.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.115563 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] receipt.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.115650 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.115803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.115900 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.116043 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.116135 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.116278 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.116363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.116510 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.116613 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.116765 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.116905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] resource.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] resource.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117241 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117337 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] resource.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117480 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117565 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] resource.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117716 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117807 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.117967 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.118074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.118201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] revoke.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.118287 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.118442 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] revoke.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.118527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.118684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.118777 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.118922 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119007 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] role.cache_time = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] role.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119254 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] role.cache_time = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119390 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] role.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119484 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] role.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119622 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] role.list_limit = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] role.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] role.list_limit = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.119950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.120111 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.120210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.120358 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.120450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.120609 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.120695 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.120857 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.120950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.121128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.121217 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.121378 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.121472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.121629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.121725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.121864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.121957 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.122102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.122186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.122347 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.122445 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.122595 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.122681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.122840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.122932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.123087 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.123263 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.123503 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.123588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.123746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.123831 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.123995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.124088 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.124245 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.124338 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.124495 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.124588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.124733 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.124828 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.124971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.125107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.125261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.125354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.125503 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.125597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.125736 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.125829 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.125971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.126056 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.126202 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.126288 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.126433 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.126519 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.126689 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.126782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.126936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127174 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127641 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] token.cache_time = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127727 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127872 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] token.cache_time = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.127962 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] token.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.128110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] token.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.128195 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.128350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.128437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] token.provider = fernet {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.128586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.128672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] token.provider = fernet {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.128823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.128909 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.129099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.129189 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.129342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.129429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.129579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.129672 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.129837 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.129943 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.130091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.130176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] trust.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.130337 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.130429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] trust.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.130574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.130666 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.130807 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.130899 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.131040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.131131 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.131285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.131372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.131516 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.131602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.131759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.131856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.132003 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.132090 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.132297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.132382 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.connection = **** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.132541 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.132637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.connection = **** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.132791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.132897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.133029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.133152 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.133294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.133386 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.133532 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.133630 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.133767 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.133865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134090 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134240 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134335 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134572 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134714 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.134951 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135045 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135188 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135272 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135422 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135521 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135661 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135826 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.135999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.136127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.136219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.136356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.136450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.136591 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.136682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.136768 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=206280) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 15:27:19.136848 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:27:19.136988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=206281) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 15:27:19.168472 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.168704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.168915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.169007 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.169173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.169400 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.177738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.178184 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.205769 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.206006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.206117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.206339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.264304 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.264989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.316152 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.316320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.316411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.316495 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.382779 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383008 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383241 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383333 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383680 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.383987 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.384114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.407398 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.407647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.407749 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.407889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.407979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408118 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408346 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408443 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408566 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408782 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.408995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.420349 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.420826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.434980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.435216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.435476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.435598 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.435689 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.435834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.435932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.436061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.436150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.436286 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.436381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=206281) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.436506 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.436725 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.436942 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=206280) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:27:19.454229 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.454439 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:27:19.454665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.454771 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.454863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:27:19.454972 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:27:19.455202 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.455297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.455392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:27:19.455474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.455663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.455759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.455927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.456022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.456189 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.456448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.467712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.467993 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.468083 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.468264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.468354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.468547 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.468643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.468859 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.477213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.477539 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.480498 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.480780 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.480874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.481112 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.481219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.481480 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.481569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.481768 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.481863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.482034 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.482131 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.482309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.482643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.482833 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.482928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.483104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.483198 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.483373 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.483460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.483638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.483738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.483905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.510833 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.512820 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.513131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.513592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.515773 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.516010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.522244 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.522380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.522518 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.522614 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.522895 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.522984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.532402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.532508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.532787 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.532915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.538016 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.538307 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.538511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.538619 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.538802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.538898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.539086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.539189 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.539371 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.539466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.539641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.539744 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.539910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.539998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.540178 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.540450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.559324 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.559604 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.564123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.567818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.568115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.568915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.569344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.573119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.575627 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.576699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.580608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.580699 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.580914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.581144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.581235 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.581498 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.581776 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.582054 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.587831 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.593318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.598416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.599286 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.599570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.599851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.600126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.600402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.601617 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.601994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.602266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.602535 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.610712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.612433 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.612712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.616441 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.616722 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.617011 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.617321 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.617600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.624153 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.624534 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.624804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.625101 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.631034 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.631316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.631598 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.631889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.632168 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.632442 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.632727 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.633112 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.634839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.635115 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.638209 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.638491 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.638774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.639067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.639347 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.650825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.652967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.653275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.653559 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.653851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.654122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.654211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.654404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.654682 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.657472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.659067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.660702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.660980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.661804 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.669287 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.670898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.673407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.674226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.674517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.674805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.675081 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.676800 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.680645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.682291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.683939 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.684220 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.684512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.684852 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.685160 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.685445 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.691992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.693537 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=206281) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.693660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.693817 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.694095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.694368 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.694640 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.694928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.695199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.695481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.695749 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.696018 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.696293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.696572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.696837 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.696932 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.697122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=206281) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.697223 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.697479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.697768 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.707052 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.707348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.707628 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.716245 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=206280) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:27:19.716527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.716808 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.717106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.717382 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.717667 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.717942 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.718228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.718497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.718764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.719032 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.719309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.719574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.719842 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=206280) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:27:19.737321 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563fb82d99c0 pid: 206281 (default app) May 09 15:27:19.740670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c308ef-f0d8-4c90-88df-bc0b501d2000 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:19.740975 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c308ef-f0d8-4c90-88df-bc0b501d2000 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:19.741289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c308ef-f0d8-4c90-88df-bc0b501d2000 None None] PATH_INFO: `/v3` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:19.745511 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon May 9 15:27:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:19.759468 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x563fb82d99c0 pid: 206280 (default app) May 09 15:27:27.691804 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0064-d299-4470-8b9c-e4cca8e4497c None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:27.692917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0064-d299-4470-8b9c-e4cca8e4497c None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:27.692917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0064-d299-4470-8b9c-e4cca8e4497c None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:27.694287 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 2/2] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:27.701582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:27.701978 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:27.702163 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:27.733669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG oslo_db.sqlalchemy.engines [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=206280) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 15:27:27.920107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG passlib.handlers.bcrypt [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=206280) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 15:27:27.920541 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG passlib.handlers.bcrypt [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=206280) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 15:27:27.975740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:27.977325 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:28.100677 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 1/3] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 15:27:28.123844 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb0064-d299-4470-8b9c-e4cca8e4497c None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:28.157460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG oslo_db.sqlalchemy.engines [None req-75fb0064-d299-4470-8b9c-e4cca8e4497c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=206281) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 15:27:28.282951 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb0064-d299-4470-8b9c-e4cca8e4497c None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:28.287190 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00939b6-df58-4b29-a368-4d58744552c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96210966ed2546dea7963761675d8627', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:28.287849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00939b6-df58-4b29-a368-4d58744552c9 admin admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:28.288133 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00939b6-df58-4b29-a368-4d58744552c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:28.288411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00939b6-df58-4b29-a368-4d58744552c9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:28.296313 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00939b6-df58-4b29-a368-4d58744552c9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:28.348152 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c00939b6-df58-4b29-a368-4d58744552c9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 15:27:28.349024 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00939b6-df58-4b29-a368-4d58744552c9 admin admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:28.358469 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 3/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 9 15:27:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 15:27:28.364921 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:28.379146 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a1c25-52df-450d-9d04-928c28d128aa None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:28.384059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eb490-947a-4e6c-8b87-9f7eb6cc5bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96210966ed2546dea7963761675d8627', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:28.385216 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb490-947a-4e6c-8b87-9f7eb6cc5bcd admin admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:28.385515 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb490-947a-4e6c-8b87-9f7eb6cc5bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:28.385804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb490-947a-4e6c-8b87-9f7eb6cc5bcd admin admin] PATH_INFO: `/v3/projects/96210966ed2546dea7963761675d8627` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:28.387346 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eb490-947a-4e6c-8b87-9f7eb6cc5bcd admin admin] RBAC: Authorizing `identity:get_project(project_id=96210966ed2546dea7963761675d8627)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:28.437897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: WARNING keystone.common.rbac_enforcer.enforcer [None req-911eb490-947a-4e6c-8b87-9f7eb6cc5bcd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 15:27:28.438353 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eb490-947a-4e6c-8b87-9f7eb6cc5bcd admin admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:28.443497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 2/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon May 9 15:27:28 2022] GET /identity/v3/projects/96210966ed2546dea7963761675d8627 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:30.637666 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0c987-fc5c-47de-8edc-8ebb9822fb12 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:30.638216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0c987-fc5c-47de-8edc-8ebb9822fb12 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:30.638216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0c987-fc5c-47de-8edc-8ebb9822fb12 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:30.642358 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 4/6] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:30.647990 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3176a1-67a9-4561-b10a-572f33a5565b None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:30.648413 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3176a1-67a9-4561-b10a-572f33a5565b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:30.648555 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3176a1-67a9-4561-b10a-572f33a5565b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:30.668946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-ae3176a1-67a9-4561-b10a-572f33a5565b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:30.670509 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-ae3176a1-67a9-4561-b10a-572f33a5565b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:30.679034 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 3/7] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 15:27:30.700678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0c987-fc5c-47de-8edc-8ebb9822fb12 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:30.712473 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0c987-fc5c-47de-8edc-8ebb9822fb12 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:30.716715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe374c3b-f5ae-4857-abe3-689d1533873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96210966ed2546dea7963761675d8627', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:30.717427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe374c3b-f5ae-4857-abe3-689d1533873a admin admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:30.717718 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe374c3b-f5ae-4857-abe3-689d1533873a admin admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:30.718001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe374c3b-f5ae-4857-abe3-689d1533873a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:30.726058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe374c3b-f5ae-4857-abe3-689d1533873a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:30.727840 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe374c3b-f5ae-4857-abe3-689d1533873a admin admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:30.736720 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 5/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 9 15:27:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 15:27:30.745751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3176a1-67a9-4561-b10a-572f33a5565b None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:30.755166 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3176a1-67a9-4561-b10a-572f33a5565b None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:30.759598 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28157444-da06-44d6-a06d-222d67d2e79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96210966ed2546dea7963761675d8627', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:30.760454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28157444-da06-44d6-a06d-222d67d2e79d admin admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:30.760735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28157444-da06-44d6-a06d-222d67d2e79d admin admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:30.761007 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28157444-da06-44d6-a06d-222d67d2e79d admin admin] PATH_INFO: `/v3/users/05c2f4bd321b4a50a016143bb5412e00` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:30.763471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28157444-da06-44d6-a06d-222d67d2e79d admin admin] RBAC: Authorizing `identity:get_user(user_id=05c2f4bd321b4a50a016143bb5412e00)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:30.765335 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28157444-da06-44d6-a06d-222d67d2e79d admin admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:30.766907 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 4/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon May 9 15:27:30 2022] GET /identity/v3/users/05c2f4bd321b4a50a016143bb5412e00 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:33.047017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea56c2-c880-47e1-be1f-2b1e53857835 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.047579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea56c2-c880-47e1-be1f-2b1e53857835 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.047579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea56c2-c880-47e1-be1f-2b1e53857835 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.048928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 6/10] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:33.054626 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f34a0-465c-4c38-be73-a7b4e5f24347 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.055119 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f34a0-465c-4c38-be73-a7b4e5f24347 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.055334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f34a0-465c-4c38-be73-a7b4e5f24347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.158053 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608f77-fd07-40f1-b1b5-0241b59eab43 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.158346 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608f77-fd07-40f1-b1b5-0241b59eab43 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.158629 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a608f77-fd07-40f1-b1b5-0241b59eab43 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.160018 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 7/11] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:33.165860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.166157 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.166443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.181296 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-0a4f34a0-465c-4c38-be73-a7b4e5f24347 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:33.182526 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-0a4f34a0-465c-4c38-be73-a7b4e5f24347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:33.206064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG passlib.handlers.bcrypt [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=206281) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 15:27:33.206496 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG passlib.handlers.bcrypt [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=206281) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 15:27:33.253724 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:33.255863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:33.260967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 5/12] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:33.284025 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f34a0-465c-4c38-be73-a7b4e5f24347 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:33.354083 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 8/13] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:33.376327 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:33.380729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f34a0-465c-4c38-be73-a7b4e5f24347 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:33.412637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77533e13-93c7-44f6-bf58-15dedc83fbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:33.413471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77533e13-93c7-44f6-bf58-15dedc83fbce None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.413765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77533e13-93c7-44f6-bf58-15dedc83fbce None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.414050 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77533e13-93c7-44f6-bf58-15dedc83fbce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.425058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77533e13-93c7-44f6-bf58-15dedc83fbce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-77533e13-93c7-44f6-bf58-15dedc83fbce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:33.436751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:33.437787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:33.438684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 6/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:27:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:27:33.445838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77533e13-93c7-44f6-bf58-15dedc83fbce None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:33.472705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16cec8-a28d-43d6-8e85-3a44abe7e06e None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:33.505937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b768f-be90-4de2-93a2-54c716a4fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:33.506839 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b768f-be90-4de2-93a2-54c716a4fa27 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.507133 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b768f-be90-4de2-93a2-54c716a4fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.507421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b768f-be90-4de2-93a2-54c716a4fa27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.540609 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77533e13-93c7-44f6-bf58-15dedc83fbce None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:33.578693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b768f-be90-4de2-93a2-54c716a4fa27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:33.580250 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b768f-be90-4de2-93a2-54c716a4fa27 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:33.581175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:33.581854 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.582137 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.582416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.583187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:33.583496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:33.584887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:33.596449 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 7/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:33.602444 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:33.675574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 9/16] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:27:33.681407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b768f-be90-4de2-93a2-54c716a4fa27 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:33.696716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0f9d4-8feb-48e5-a8b2-c02caba9419c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:33.730011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8a1b-3e9a-45ef-b263-d3406b7ff3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:33.730872 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8a1b-3e9a-45ef-b263-d3406b7ff3de None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.731157 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8a1b-3e9a-45ef-b263-d3406b7ff3de None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.731431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8a1b-3e9a-45ef-b263-d3406b7ff3de None admin] PATH_INFO: `/v3/users/05c2f4bd321b4a50a016143bb5412e00` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.732892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8a1b-3e9a-45ef-b263-d3406b7ff3de None admin] RBAC: Authorizing `identity:get_user(user_id=05c2f4bd321b4a50a016143bb5412e00)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:33.734368 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e8a1b-3e9a-45ef-b263-d3406b7ff3de None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:33.735968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 8/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:33 2022] GET /identity/v3/users/05c2f4bd321b4a50a016143bb5412e00 => generated 247 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:33.741800 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8a1b-3e9a-45ef-b263-d3406b7ff3de None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:33.775943 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b768f-be90-4de2-93a2-54c716a4fa27 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:33.808078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:33.808873 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.809194 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.809481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.834543 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8a1b-3e9a-45ef-b263-d3406b7ff3de None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:33.870908 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602ab2-ee4a-42d3-95da-c92b188a9780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:33.871723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602ab2-ee4a-42d3-95da-c92b188a9780 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:33.872005 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602ab2-ee4a-42d3-95da-c92b188a9780 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:33.872280 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602ab2-ee4a-42d3-95da-c92b188a9780 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:33.873594 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602ab2-ee4a-42d3-95da-c92b188a9780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:33.875083 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602ab2-ee4a-42d3-95da-c92b188a9780 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:33.876767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:33.882681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602ab2-ee4a-42d3-95da-c92b188a9780 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:33.936990 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:33.938405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:27:33.959417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:33.960375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:33.961316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:27:33.961316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:27:33.961316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:27:33.961316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:27:33.961316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:27:33.966614 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:33.980331 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602ab2-ee4a-42d3-95da-c92b188a9780 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:34.012206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6435c54-69f0-4593-975d-727d146bc2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:34.012880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6435c54-69f0-4593-975d-727d146bc2ee None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:34.013177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6435c54-69f0-4593-975d-727d146bc2ee None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:34.013453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6435c54-69f0-4593-975d-727d146bc2ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:34.014288 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6435c54-69f0-4593-975d-727d146bc2ee None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.domain.id=default, user.id=05c2f4bd321b4a50a016143bb5412e00) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:34.014599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6435c54-69f0-4593-975d-727d146bc2ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:34.016034 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6435c54-69f0-4593-975d-727d146bc2ee None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:34.023059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 9 15:27:33 2022] GET /identity/v3/role_assignments?user.id=05c2f4bd321b4a50a016143bb5412e00&scope.domain.id=default&role.id=58c34600086c469b87bfa39b74dd2607 => generated 233 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:34.065547 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fc15e-4c29-4d02-8b44-126af827126b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:34.101639 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe93cc18-168f-45b5-848b-ae58c779efad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:34.102329 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93cc18-168f-45b5-848b-ae58c779efad None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:34.102619 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93cc18-168f-45b5-848b-ae58c779efad None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:34.102902 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93cc18-168f-45b5-848b-ae58c779efad None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:34.103661 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93cc18-168f-45b5-848b-ae58c779efad None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:34.103973 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93cc18-168f-45b5-848b-ae58c779efad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:34.105455 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe93cc18-168f-45b5-848b-ae58c779efad None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:34.124514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 11/21] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:36.577100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54ba43-082b-40d7-af88-d2395fe54dbd None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:36.577647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54ba43-082b-40d7-af88-d2395fe54dbd None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:36.577647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54ba43-082b-40d7-af88-d2395fe54dbd None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:36.579339 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:36.590683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d8bd2-1370-445e-8317-6c2a4e2be172 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:36.591530 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d8bd2-1370-445e-8317-6c2a4e2be172 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:36.591980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d8bd2-1370-445e-8317-6c2a4e2be172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:36.614609 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-b85d8bd2-1370-445e-8317-6c2a4e2be172 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:36.616180 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-b85d8bd2-1370-445e-8317-6c2a4e2be172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:36.691640 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:36.722159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54ba43-082b-40d7-af88-d2395fe54dbd None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:36.849941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54ba43-082b-40d7-af88-d2395fe54dbd None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:36.896545 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa741b-7d10-46aa-a98e-e2f98cf17166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:36.897409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa741b-7d10-46aa-a98e-e2f98cf17166 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:36.897700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa741b-7d10-46aa-a98e-e2f98cf17166 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:36.897984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa741b-7d10-46aa-a98e-e2f98cf17166 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:36.914409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa741b-7d10-46aa-a98e-e2f98cf17166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-5ffa741b-7d10-46aa-a98e-e2f98cf17166 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:36.922086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:36.923208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:36.924160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 12/24] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:27:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:36.934250 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d8bd2-1370-445e-8317-6c2a4e2be172 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:37.046359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ed7b2-7f13-4cf3-9e43-9b36b06dc7a8 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.046660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ed7b2-7f13-4cf3-9e43-9b36b06dc7a8 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.046951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ed7b2-7f13-4cf3-9e43-9b36b06dc7a8 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.048935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 13/25] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:37.058349 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab7b36-30c8-49ab-a2b4-b1a028def5b5 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.058641 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab7b36-30c8-49ab-a2b4-b1a028def5b5 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.058922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab7b36-30c8-49ab-a2b4-b1a028def5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.099442 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-53ab7b36-30c8-49ab-a2b4-b1a028def5b5 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:37.100903 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-53ab7b36-30c8-49ab-a2b4-b1a028def5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:37.113064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d8bd2-1370-445e-8317-6c2a4e2be172 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:37.188687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:37.189308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.189607 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.190016 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.190710 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:37.191032 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:37.192733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:37.198432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 14/26] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:37.215389 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 13/27] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:37.221349 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab7b36-30c8-49ab-a2b4-b1a028def5b5 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:37.234161 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:37.423092 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab7b36-30c8-49ab-a2b4-b1a028def5b5 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:37.458677 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89715a4b-4bd0-4927-987a-19077e9a1340 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:37.492627 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1764-61a2-41e1-b466-45e12de6ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:37.493808 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1764-61a2-41e1-b466-45e12de6ae54 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.496912 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1764-61a2-41e1-b466-45e12de6ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.497384 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab1764-61a2-41e1-b466-45e12de6ae54 None admin] PATH_INFO: `/v3/users/05c2f4bd321b4a50a016143bb5412e00` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.499700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab1764-61a2-41e1-b466-45e12de6ae54 None admin] RBAC: Authorizing `identity:get_user(user_id=05c2f4bd321b4a50a016143bb5412e00)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:37.501352 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab1764-61a2-41e1-b466-45e12de6ae54 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:37.503332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 15/28] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:37 2022] GET /identity/v3/users/05c2f4bd321b4a50a016143bb5412e00 => generated 247 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:37.509992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1764-61a2-41e1-b466-45e12de6ae54 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:37.520713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffede2-1d17-4a74-8cd7-d92c97a3f993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:37.521647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffede2-1d17-4a74-8cd7-d92c97a3f993 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.522929 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffede2-1d17-4a74-8cd7-d92c97a3f993 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.523374 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffede2-1d17-4a74-8cd7-d92c97a3f993 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.524329 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffede2-1d17-4a74-8cd7-d92c97a3f993 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:37.525939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffede2-1d17-4a74-8cd7-d92c97a3f993 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:37.629170 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab1764-61a2-41e1-b466-45e12de6ae54 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:37.675644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 14/29] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:27:37 2022] POST /identity/v3/roles => generated 219 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:27:37.676592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe83fb5-64f5-40b1-98e3-8f7e69c3f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:37.677605 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83fb5-64f5-40b1-98e3-8f7e69c3f89e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.677901 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83fb5-64f5-40b1-98e3-8f7e69c3f89e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.678186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe83fb5-64f5-40b1-98e3-8f7e69c3f89e None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.678529 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ed614-0bee-4567-9364-216df1a3f9f0 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.679005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ed614-0bee-4567-9364-216df1a3f9f0 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.679456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ed614-0bee-4567-9364-216df1a3f9f0 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.679578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe83fb5-64f5-40b1-98e3-8f7e69c3f89e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:37.680965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe83fb5-64f5-40b1-98e3-8f7e69c3f89e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:37.681771 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 15/30] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:37.683248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 16/31] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:37.689007 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe83fb5-64f5-40b1-98e3-8f7e69c3f89e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:37.689935 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53adf4-1c50-4cbe-8098-df476c83f554 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.690238 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53adf4-1c50-4cbe-8098-df476c83f554 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.690594 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53adf4-1c50-4cbe-8098-df476c83f554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.710711 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-5c53adf4-1c50-4cbe-8098-df476c83f554 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:37.712740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-5c53adf4-1c50-4cbe-8098-df476c83f554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:37.798021 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 16/32] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:37.799823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b093f-6d90-4cd0-a409-10b997a1a467 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.800294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b093f-6d90-4cd0-a409-10b997a1a467 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.800465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe83fb5-64f5-40b1-98e3-8f7e69c3f89e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:37.800900 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b093f-6d90-4cd0-a409-10b997a1a467 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.803483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 17/33] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:37.814610 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd6b7-8d25-4f2e-8cfb-7a1cec1bfc62 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.815279 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd6b7-8d25-4f2e-8cfb-7a1cec1bfc62 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.815798 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd6b7-8d25-4f2e-8cfb-7a1cec1bfc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.840968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba814fa-c267-4b1a-9b78-46014fef0cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:37.841985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba814fa-c267-4b1a-9b78-46014fef0cf4 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:37.842271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba814fa-c267-4b1a-9b78-46014fef0cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:37.842612 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba814fa-c267-4b1a-9b78-46014fef0cf4 None admin] PATH_INFO: `/v3/domains/default/users/05c2f4bd321b4a50a016143bb5412e00/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:37.844289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-058bd6b7-8d25-4f2e-8cfb-7a1cec1bfc62 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:37.844910 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba814fa-c267-4b1a-9b78-46014fef0cf4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=05c2f4bd321b4a50a016143bb5412e00, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:37.846046 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-058bd6b7-8d25-4f2e-8cfb-7a1cec1bfc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:37.846544 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba814fa-c267-4b1a-9b78-46014fef0cf4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:37.927726 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 18/34] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:37.928550 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bd6b7-8d25-4f2e-8cfb-7a1cec1bfc62 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:37.981418 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 17/35] 10.222.0.13 () {58 vars in 1361 bytes} [Mon May 9 15:27:37 2022] PUT /identity/v3/domains/default/users/05c2f4bd321b4a50a016143bb5412e00/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:27:37.983781 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba814fa-c267-4b1a-9b78-46014fef0cf4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:38.041515 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bd6b7-8d25-4f2e-8cfb-7a1cec1bfc62 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:38.078800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52412f2b-9b28-40c1-8c3f-cb3c8a7e95ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:38.079665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412f2b-9b28-40c1-8c3f-cb3c8a7e95ce None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.079801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412f2b-9b28-40c1-8c3f-cb3c8a7e95ce None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.080041 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412f2b-9b28-40c1-8c3f-cb3c8a7e95ce None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.080815 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52412f2b-9b28-40c1-8c3f-cb3c8a7e95ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:38.082450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52412f2b-9b28-40c1-8c3f-cb3c8a7e95ce None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:38.103584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba814fa-c267-4b1a-9b78-46014fef0cf4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:38.111270 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 19/36] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:27:37 2022] POST /identity/v3/roles => generated 225 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:27:38.159130 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:38.160019 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.160324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.160610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.174534 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:38.176285 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:27:38.198989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:38.199943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:38.200818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:27:38.200818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:27:38.200818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:27:38.200818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:38.200818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 18/37] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:38.207006 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52412f2b-9b28-40c1-8c3f-cb3c8a7e95ce None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:38.326234 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52412f2b-9b28-40c1-8c3f-cb3c8a7e95ce None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:38.364605 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc0be8-b09f-4500-a741-395736b25b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:38.365331 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc0be8-b09f-4500-a741-395736b25b33 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.365623 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc0be8-b09f-4500-a741-395736b25b33 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.365912 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc0be8-b09f-4500-a741-395736b25b33 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.366671 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc0be8-b09f-4500-a741-395736b25b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:38.366993 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc0be8-b09f-4500-a741-395736b25b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:38.368410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc0be8-b09f-4500-a741-395736b25b33 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:38.382700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 20/38] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:38.388283 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:38.489266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c5d58-e0ab-4027-b252-587d0cb2f194 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.489570 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c5d58-e0ab-4027-b252-587d0cb2f194 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.489759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c5d58-e0ab-4027-b252-587d0cb2f194 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.491396 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 21/39] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:38.496584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36ddb-9f72-4bd6-89bf-bcbf74e374cb None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.496881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36ddb-9f72-4bd6-89bf-bcbf74e374cb None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.497199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36ddb-9f72-4bd6-89bf-bcbf74e374cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.509186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea5e07-72f6-4da8-a951-8c370a055d14 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:38.523263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-3fb36ddb-9f72-4bd6-89bf-bcbf74e374cb None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:38.524515 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-3fb36ddb-9f72-4bd6-89bf-bcbf74e374cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:38.548653 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5328c-4175-40f0-83ba-35d6cc4f6b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:38.549746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5328c-4175-40f0-83ba-35d6cc4f6b88 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.550258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5328c-4175-40f0-83ba-35d6cc4f6b88 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.550765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5328c-4175-40f0-83ba-35d6cc4f6b88 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.551716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5328c-4175-40f0-83ba-35d6cc4f6b88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:38.553472 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5328c-4175-40f0-83ba-35d6cc4f6b88 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:38.589508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 19/40] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:27:38 2022] POST /identity/v3/projects => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:27:38.603383 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 22/41] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:38.628530 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5328c-4175-40f0-83ba-35d6cc4f6b88 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:38.710004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb77a15-8e91-4e8c-a227-b49730a11804 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.711005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb77a15-8e91-4e8c-a227-b49730a11804 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.711512 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb77a15-8e91-4e8c-a227-b49730a11804 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.713663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 23/42] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:38.721310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74073d46-ead5-43cc-a735-a9d50c7ef328 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.721657 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74073d46-ead5-43cc-a735-a9d50c7ef328 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.721884 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74073d46-ead5-43cc-a735-a9d50c7ef328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.740402 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-74073d46-ead5-43cc-a735-a9d50c7ef328 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:38.741575 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-74073d46-ead5-43cc-a735-a9d50c7ef328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:38.756428 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5328c-4175-40f0-83ba-35d6cc4f6b88 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:38.794996 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981bb8a-882c-43a0-af42-f150a2113f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:38.795626 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981bb8a-882c-43a0-af42-f150a2113f53 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.795921 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981bb8a-882c-43a0-af42-f150a2113f53 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.796231 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981bb8a-882c-43a0-af42-f150a2113f53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.798406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981bb8a-882c-43a0-af42-f150a2113f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:38.799174 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981bb8a-882c-43a0-af42-f150a2113f53 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:38.800855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 20/43] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:38.807313 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981bb8a-882c-43a0-af42-f150a2113f53 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:38.824066 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 24/44] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:38.847057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74073d46-ead5-43cc-a735-a9d50c7ef328 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:38.921595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981bb8a-882c-43a0-af42-f150a2113f53 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:38.953857 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74073d46-ead5-43cc-a735-a9d50c7ef328 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:38.954940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a6257-b1c0-474a-b1ab-c12e5328cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:38.955633 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6257-b1c0-474a-b1ab-c12e5328cefe None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.955921 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6257-b1c0-474a-b1ab-c12e5328cefe None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.956203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6257-b1c0-474a-b1ab-c12e5328cefe None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.957115 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a6257-b1c0-474a-b1ab-c12e5328cefe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:38.959147 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a6257-b1c0-474a-b1ab-c12e5328cefe None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:38.984085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 21/45] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:27:38 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:27:38.993155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6410b5-cc49-4af7-a204-d697af35a944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:38.994172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6410b5-cc49-4af7-a204-d697af35a944 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:38.994669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6410b5-cc49-4af7-a204-d697af35a944 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:38.995191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6410b5-cc49-4af7-a204-d697af35a944 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:38.996199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6410b5-cc49-4af7-a204-d697af35a944 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:38.997936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6410b5-cc49-4af7-a204-d697af35a944 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:39.014374 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 25/46] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:27:38 2022] POST /identity/v3/roles => generated 223 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:27:40.576112 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5045a-9e4a-4be9-bd6b-b7d32e352363 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:40.576708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5045a-9e4a-4be9-bd6b-b7d32e352363 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:40.576708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5045a-9e4a-4be9-bd6b-b7d32e352363 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:40.578120 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 22/47] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:40.583608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b003f-0d16-4670-8881-6edbb59a30ad None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:40.584051 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b003f-0d16-4670-8881-6edbb59a30ad None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:40.584193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b003f-0d16-4670-8881-6edbb59a30ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:40.601119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-4a5b003f-0d16-4670-8881-6edbb59a30ad None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:40.602325 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-4a5b003f-0d16-4670-8881-6edbb59a30ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:40.666746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 26/48] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:40.689892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5045a-9e4a-4be9-bd6b-b7d32e352363 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:40.793342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5045a-9e4a-4be9-bd6b-b7d32e352363 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:40.826670 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ed8c5-83b7-4a39-86a6-cdf32d47c8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:40.827395 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ed8c5-83b7-4a39-86a6-cdf32d47c8ce None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:40.827681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ed8c5-83b7-4a39-86a6-cdf32d47c8ce None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:40.827963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ed8c5-83b7-4a39-86a6-cdf32d47c8ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:40.841888 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ed8c5-83b7-4a39-86a6-cdf32d47c8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-3c9ed8c5-83b7-4a39-86a6-cdf32d47c8ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:40.850144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:40.851452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:40.852423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 23/49] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:27:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:40.857596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b003f-0d16-4670-8881-6edbb59a30ad None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:40.951168 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b003f-0d16-4670-8881-6edbb59a30ad None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:40.984458 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:40.985165 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:40.985459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:40.985741 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:40.986505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:40.986820 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:40.988226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:40.996282 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 27/50] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:41.001766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ed8c5-83b7-4a39-86a6-cdf32d47c8ce None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:41.104857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ed8c5-83b7-4a39-86a6-cdf32d47c8ce None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:41.141028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99048a-607f-4d7b-bb44-3095131c1f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:41.141944 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99048a-607f-4d7b-bb44-3095131c1f77 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:41.142231 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99048a-607f-4d7b-bb44-3095131c1f77 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:41.142535 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99048a-607f-4d7b-bb44-3095131c1f77 None admin] PATH_INFO: `/v3/users/05c2f4bd321b4a50a016143bb5412e00` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:41.143773 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99048a-607f-4d7b-bb44-3095131c1f77 None admin] RBAC: Authorizing `identity:get_user(user_id=05c2f4bd321b4a50a016143bb5412e00)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:41.145832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99048a-607f-4d7b-bb44-3095131c1f77 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:41.147504 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 24/51] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:41 2022] GET /identity/v3/users/05c2f4bd321b4a50a016143bb5412e00 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:41.153085 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:41.247430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0f29a-ebeb-4a21-b4b6-9925c0f27341 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:41.250670 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9d7a-097d-471a-bc51-f037d2b45e43 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:41.250961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9d7a-097d-471a-bc51-f037d2b45e43 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:41.251246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9d7a-097d-471a-bc51-f037d2b45e43 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:41.252630 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 25/52] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:41.258575 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd9b5-63f2-41e3-92ad-42c9afaaab88 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:41.258868 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd9b5-63f2-41e3-92ad-42c9afaaab88 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:41.259155 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd9b5-63f2-41e3-92ad-42c9afaaab88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:41.276100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-92acd9b5-63f2-41e3-92ad-42c9afaaab88 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:41.277661 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-92acd9b5-63f2-41e3-92ad-42c9afaaab88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:41.281166 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec1382-b04f-4ba9-a083-12be98fef563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:41.281970 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec1382-b04f-4ba9-a083-12be98fef563 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:41.282264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec1382-b04f-4ba9-a083-12be98fef563 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:41.282547 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec1382-b04f-4ba9-a083-12be98fef563 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:41.283988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec1382-b04f-4ba9-a083-12be98fef563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:41.285473 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec1382-b04f-4ba9-a083-12be98fef563 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:41.287096 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 28/53] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:41.292660 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec1382-b04f-4ba9-a083-12be98fef563 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:41.341249 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 26/54] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:41.364167 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd9b5-63f2-41e3-92ad-42c9afaaab88 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:41.386853 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec1382-b04f-4ba9-a083-12be98fef563 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:41.420594 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:41.421299 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:41.421588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:41.421872 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:41.422729 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.domain.id=default, user.id=05c2f4bd321b4a50a016143bb5412e00) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:41.423048 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:41.424499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:41.433649 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 29/55] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 9 15:27:41 2022] GET /identity/v3/role_assignments?user.id=05c2f4bd321b4a50a016143bb5412e00&scope.domain.id=default&role.id=58c34600086c469b87bfa39b74dd2607 => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:41.460141 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd9b5-63f2-41e3-92ad-42c9afaaab88 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:41.495900 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b138a-deed-46b5-a1d3-06b420d6ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:41.496729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b138a-deed-46b5-a1d3-06b420d6ed2f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:41.497017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b138a-deed-46b5-a1d3-06b420d6ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:41.497326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b138a-deed-46b5-a1d3-06b420d6ed2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:41.498631 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b138a-deed-46b5-a1d3-06b420d6ed2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:41.500141 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b138a-deed-46b5-a1d3-06b420d6ed2f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:41.501909 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 27/56] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:41.508395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:41.608935 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdd57f-20d5-4be8-ad17-61a22a4fe174 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:41.643342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90203824-dcb7-4481-a671-abde1d28122d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:41.643950 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90203824-dcb7-4481-a671-abde1d28122d None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:41.644243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90203824-dcb7-4481-a671-abde1d28122d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:41.644528 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90203824-dcb7-4481-a671-abde1d28122d None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:41.645309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90203824-dcb7-4481-a671-abde1d28122d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:41.646794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90203824-dcb7-4481-a671-abde1d28122d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:41.673409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 30/57] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 9 15:27:41 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:27:43.874903 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01399fc7-d166-4cae-92bb-f2a484d54915 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:43.875453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01399fc7-d166-4cae-92bb-f2a484d54915 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:43.875453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01399fc7-d166-4cae-92bb-f2a484d54915 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:43.876855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 28/58] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:43.882730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1428a-5725-4d8c-8e35-9340f9178810 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:43.883166 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1428a-5725-4d8c-8e35-9340f9178810 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:43.883300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1428a-5725-4d8c-8e35-9340f9178810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:43.899557 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-f1d1428a-5725-4d8c-8e35-9340f9178810 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:43.900740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-f1d1428a-5725-4d8c-8e35-9340f9178810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:43.963581 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 31/59] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:43.987639 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01399fc7-d166-4cae-92bb-f2a484d54915 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:44.083330 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01399fc7-d166-4cae-92bb-f2a484d54915 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:44.117324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee85ad-dd49-495e-ac61-a9242e0ee5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:44.118121 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee85ad-dd49-495e-ac61-a9242e0ee5cd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:44.118412 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee85ad-dd49-495e-ac61-a9242e0ee5cd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:44.118698 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee85ad-dd49-495e-ac61-a9242e0ee5cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:44.119898 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee85ad-dd49-495e-ac61-a9242e0ee5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:44.121408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee85ad-dd49-495e-ac61-a9242e0ee5cd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:44.123045 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 29/60] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:44.128551 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1428a-5725-4d8c-8e35-9340f9178810 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:44.221212 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1428a-5725-4d8c-8e35-9340f9178810 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:44.253981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbf50d-3219-4a5c-b504-95e9f8287b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:44.254652 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbf50d-3219-4a5c-b504-95e9f8287b2e None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:44.254942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbf50d-3219-4a5c-b504-95e9f8287b2e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:44.255226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbf50d-3219-4a5c-b504-95e9f8287b2e None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:44.255943 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbf50d-3219-4a5c-b504-95e9f8287b2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:44.257427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbf50d-3219-4a5c-b504-95e9f8287b2e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:44.281405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 32/61] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:27:44 2022] POST /identity/v3/users => generated 275 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:27:47.646346 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71866fe6-a9a2-46ec-bd1c-62e96557c70e None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:47.646905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71866fe6-a9a2-46ec-bd1c-62e96557c70e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:47.646905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71866fe6-a9a2-46ec-bd1c-62e96557c70e None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:47.648615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 30/62] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:47.654119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2e4b-d666-4713-bc8f-e055a566a230 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:47.654651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2e4b-d666-4713-bc8f-e055a566a230 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:47.654741 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2e4b-d666-4713-bc8f-e055a566a230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:47.686328 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-df6f2e4b-d666-4713-bc8f-e055a566a230 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:47.687663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-df6f2e4b-d666-4713-bc8f-e055a566a230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:47.711559 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efefd1-cab4-4a86-81bf-713fc5889beb None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:47.711834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efefd1-cab4-4a86-81bf-713fc5889beb None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:47.712115 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efefd1-cab4-4a86-81bf-713fc5889beb None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:47.713532 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 31/63] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:47.720464 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9b5a0-781c-4c54-af46-0e7232decc2d None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:47.720755 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9b5a0-781c-4c54-af46-0e7232decc2d None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:47.721058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9b5a0-781c-4c54-af46-0e7232decc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:47.756869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-f8a9b5a0-781c-4c54-af46-0e7232decc2d None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:47.758110 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-f8a9b5a0-781c-4c54-af46-0e7232decc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:47.794866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 33/64] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:47.817712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f2e4b-d666-4713-bc8f-e055a566a230 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:47.885136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 32/65] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:47.897819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1f57d-b025-4811-b1ef-f6b7cab9c987 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:47.898443 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1f57d-b025-4811-b1ef-f6b7cab9c987 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:47.898947 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1f57d-b025-4811-b1ef-f6b7cab9c987 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:47.900555 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 33/66] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:47.918650 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181e4e2-ee3a-4261-a41b-d1cc3f7f122b None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:47.919258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181e4e2-ee3a-4261-a41b-d1cc3f7f122b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:47.919774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181e4e2-ee3a-4261-a41b-d1cc3f7f122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:47.925933 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f2e4b-d666-4713-bc8f-e055a566a230 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:47.943301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-d181e4e2-ee3a-4261-a41b-d1cc3f7f122b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:47.944857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-d181e4e2-ee3a-4261-a41b-d1cc3f7f122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:47.962212 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e5550-f2e5-4c36-a386-f66a2bfe9e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:47.963023 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e5550-f2e5-4c36-a386-f66a2bfe9e83 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:47.963315 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e5550-f2e5-4c36-a386-f66a2bfe9e83 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:47.963601 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e5550-f2e5-4c36-a386-f66a2bfe9e83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:47.975402 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e5550-f2e5-4c36-a386-f66a2bfe9e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-d27e5550-f2e5-4c36-a386-f66a2bfe9e83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:47.984875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:47.986059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:47.987123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:27:47.989386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 34/67] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:27:47.990588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e5550-f2e5-4c36-a386-f66a2bfe9e83 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.021128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 34/68] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:48.023176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181e4e2-ee3a-4261-a41b-d1cc3f7f122b None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.110830 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e5550-f2e5-4c36-a386-f66a2bfe9e83 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.145952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf18667-1391-4c34-bd20-e944f8291bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.147026 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf18667-1391-4c34-bd20-e944f8291bf5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.147479 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf18667-1391-4c34-bd20-e944f8291bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.147989 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf18667-1391-4c34-bd20-e944f8291bf5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.148879 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181e4e2-ee3a-4261-a41b-d1cc3f7f122b None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.158777 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf18667-1391-4c34-bd20-e944f8291bf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5bf18667-1391-4c34-bd20-e944f8291bf5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:48.166790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:48.167730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:48.168651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:27:48.171200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 35/69] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:27:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:48.172231 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf18667-1391-4c34-bd20-e944f8291bf5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.189339 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.190031 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.190328 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.190662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.191381 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:48.191701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:48.193695 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:48.202696 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 35/70] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:48.203686 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.281466 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf18667-1391-4c34-bd20-e944f8291bf5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.307172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a412-94ec-4303-adbf-07f606feac97 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.325839 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8857d-13e5-481a-ba43-ab2daf2ac896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.327040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8857d-13e5-481a-ba43-ab2daf2ac896 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.327899 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8857d-13e5-481a-ba43-ab2daf2ac896 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.328351 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8857d-13e5-481a-ba43-ab2daf2ac896 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.339269 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8857d-13e5-481a-ba43-ab2daf2ac896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:48.340680 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd8952-eb30-430d-829b-e81fe697b077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.341384 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd8952-eb30-430d-829b-e81fe697b077 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.341673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd8952-eb30-430d-829b-e81fe697b077 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.341953 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd8952-eb30-430d-829b-e81fe697b077 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.342718 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd8952-eb30-430d-829b-e81fe697b077 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:48.343033 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd8952-eb30-430d-829b-e81fe697b077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:48.344429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd8952-eb30-430d-829b-e81fe697b077 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-0ed8857d-13e5-481a-ba43-ab2daf2ac896 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:48.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:48.349913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:27:48.350968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:27:48.353067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 36/71] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:48.353277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 36/72] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:48.354145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8857d-13e5-481a-ba43-ab2daf2ac896 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.354517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84906db3-6353-48dc-8cb6-6b171eb780bd None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.354807 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84906db3-6353-48dc-8cb6-6b171eb780bd None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.355086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84906db3-6353-48dc-8cb6-6b171eb780bd None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.356486 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 37/73] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:48.361610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84906db3-6353-48dc-8cb6-6b171eb780bd None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.460888 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84906db3-6353-48dc-8cb6-6b171eb780bd None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.467144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8857d-13e5-481a-ba43-ab2daf2ac896 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.493336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65031aa4-eca0-45e7-aca7-0f691ed8608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.494205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65031aa4-eca0-45e7-aca7-0f691ed8608b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.494489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65031aa4-eca0-45e7-aca7-0f691ed8608b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.494767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65031aa4-eca0-45e7-aca7-0f691ed8608b None admin] PATH_INFO: `/v3/users/05c2f4bd321b4a50a016143bb5412e00` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.495985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65031aa4-eca0-45e7-aca7-0f691ed8608b None admin] RBAC: Authorizing `identity:get_user(user_id=05c2f4bd321b4a50a016143bb5412e00)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:48.497473 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65031aa4-eca0-45e7-aca7-0f691ed8608b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:48.499107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 38/74] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/users/05c2f4bd321b4a50a016143bb5412e00 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:48.500233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb6470-7883-41a8-b84d-5d579c1e0c55 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.500518 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb6470-7883-41a8-b84d-5d579c1e0c55 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.500797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb6470-7883-41a8-b84d-5d579c1e0c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.502363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639af1c-5cc9-4e19-84dc-405f3bb8256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.503542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639af1c-5cc9-4e19-84dc-405f3bb8256e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.503990 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639af1c-5cc9-4e19-84dc-405f3bb8256e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.504301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639af1c-5cc9-4e19-84dc-405f3bb8256e None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.516874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-18bb6470-7883-41a8-b84d-5d579c1e0c55 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:48.518101 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-18bb6470-7883-41a8-b84d-5d579c1e0c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:48.519847 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639af1c-5cc9-4e19-84dc-405f3bb8256e None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:48.521426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639af1c-5cc9-4e19-84dc-405f3bb8256e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:48.523103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 37/75] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:48.523880 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639af1c-5cc9-4e19-84dc-405f3bb8256e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.583353 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 39/76] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:48.584834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637578e-0bc7-4a82-8933-4dc75a3a2ffa None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.585238 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637578e-0bc7-4a82-8933-4dc75a3a2ffa None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.585527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637578e-0bc7-4a82-8933-4dc75a3a2ffa None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.587402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 40/77] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:48.587847 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637578e-0bc7-4a82-8933-4dc75a3a2ffa None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.622055 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639af1c-5cc9-4e19-84dc-405f3bb8256e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.653608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.654315 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.654856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.654856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.655638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:48.655957 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:48.657373 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:48.665755 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 38/78] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:48.666474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.685871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637578e-0bc7-4a82-8933-4dc75a3a2ffa None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.717661 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f1714-699c-49ba-bc93-a8f8dab1bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.718249 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1714-699c-49ba-bc93-a8f8dab1bb01 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.718543 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1714-699c-49ba-bc93-a8f8dab1bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.718818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f1714-699c-49ba-bc93-a8f8dab1bb01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.760266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120e08-0e4f-4ee0-821e-65777213dc39 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:48.781794 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f1714-699c-49ba-bc93-a8f8dab1bb01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:48.783257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f1714-699c-49ba-bc93-a8f8dab1bb01 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:48.792066 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51814ed0-d017-4558-84d7-839cbc3c1cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:48.792723 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51814ed0-d017-4558-84d7-839cbc3c1cfe None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.793012 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51814ed0-d017-4558-84d7-839cbc3c1cfe None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.793329 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51814ed0-d017-4558-84d7-839cbc3c1cfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.860936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51814ed0-d017-4558-84d7-839cbc3c1cfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:48.862476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51814ed0-d017-4558-84d7-839cbc3c1cfe None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:48.877667 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 41/79] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:27:48.878958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c95b-9e5e-45ce-b5b9-544a8820af7a None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:48.879253 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c95b-9e5e-45ce-b5b9-544a8820af7a None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:48.879537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651c95b-9e5e-45ce-b5b9-544a8820af7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:48.895404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-e651c95b-9e5e-45ce-b5b9-544a8820af7a None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:48.896587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-e651c95b-9e5e-45ce-b5b9-544a8820af7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:48.959263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 39/80] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:27:48.959936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51814ed0-d017-4558-84d7-839cbc3c1cfe None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:48.965641 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 42/81] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:48.966508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651c95b-9e5e-45ce-b5b9-544a8820af7a None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.058478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51814ed0-d017-4558-84d7-839cbc3c1cfe None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.066829 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651c95b-9e5e-45ce-b5b9-544a8820af7a None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.090812 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76750-5094-45be-8723-e120e010ad41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.091613 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76750-5094-45be-8723-e120e010ad41 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.091907 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76750-5094-45be-8723-e120e010ad41 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.092192 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76750-5094-45be-8723-e120e010ad41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.099440 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5097e0-07fb-42c4-a96d-c8ecb23ad7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.100306 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5097e0-07fb-42c4-a96d-c8ecb23ad7e4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.100587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5097e0-07fb-42c4-a96d-c8ecb23ad7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.100863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5097e0-07fb-42c4-a96d-c8ecb23ad7e4 None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.102126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5097e0-07fb-42c4-a96d-c8ecb23ad7e4 None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.102809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76750-5094-45be-8723-e120e010ad41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:49.103594 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5097e0-07fb-42c4-a96d-c8ecb23ad7e4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.105255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 43/82] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:49.106019 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5097e0-07fb-42c4-a96d-c8ecb23ad7e4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-aeb76750-5094-45be-8723-e120e010ad41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:49.111485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:49.112634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:27:49.113522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 40/83] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:49.114322 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76750-5094-45be-8723-e120e010ad41 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.204237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5097e0-07fb-42c4-a96d-c8ecb23ad7e4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.207222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76750-5094-45be-8723-e120e010ad41 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.237604 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f95b-a0f5-4604-875e-ee17a62f7ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.238457 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f95b-a0f5-4604-875e-ee17a62f7ec6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.238748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f95b-a0f5-4604-875e-ee17a62f7ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.239028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f95b-a0f5-4604-875e-ee17a62f7ec6 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.239196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a47e89-de17-4b62-977f-c3b94d18d250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.240042 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47e89-de17-4b62-977f-c3b94d18d250 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.240223 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668f95b-a0f5-4604-875e-ee17a62f7ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.240333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47e89-de17-4b62-977f-c3b94d18d250 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.240607 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47e89-de17-4b62-977f-c3b94d18d250 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.241728 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668f95b-a0f5-4604-875e-ee17a62f7ec6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.241848 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a47e89-de17-4b62-977f-c3b94d18d250 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.243244 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a47e89-de17-4b62-977f-c3b94d18d250 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.243729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 44/84] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:49.244393 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f95b-a0f5-4604-875e-ee17a62f7ec6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.245019 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 41/85] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:49.245715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a47e89-de17-4b62-977f-c3b94d18d250 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.337752 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f95b-a0f5-4604-875e-ee17a62f7ec6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.340989 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a47e89-de17-4b62-977f-c3b94d18d250 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.369429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b7853-4fb7-42cc-a3c5-2ba1212ec5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.370207 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b7853-4fb7-42cc-a3c5-2ba1212ec5aa None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.370539 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b7853-4fb7-42cc-a3c5-2ba1212ec5aa None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.370817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b7853-4fb7-42cc-a3c5-2ba1212ec5aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.371943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b7853-4fb7-42cc-a3c5-2ba1212ec5aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.372691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259fa65-5d44-4b27-a9db-50680f9d36d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.373378 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259fa65-5d44-4b27-a9db-50680f9d36d7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.373496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b7853-4fb7-42cc-a3c5-2ba1212ec5aa None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.373674 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259fa65-5d44-4b27-a9db-50680f9d36d7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.373963 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259fa65-5d44-4b27-a9db-50680f9d36d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.375025 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 45/86] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:49.380751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b7853-4fb7-42cc-a3c5-2ba1212ec5aa None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.439415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259fa65-5d44-4b27-a9db-50680f9d36d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.440886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259fa65-5d44-4b27-a9db-50680f9d36d7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.473617 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b7853-4fb7-42cc-a3c5-2ba1212ec5aa None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.505789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.506455 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.506742 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.507021 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.507776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:49.508089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.509508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.517343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 46/87] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:49.518130 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.536242 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 42/88] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:27:49.537420 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259fa65-5d44-4b27-a9db-50680f9d36d7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.619162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c6cfd-4a97-49b3-824a-0eaeba74ba91 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.629898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259fa65-5d44-4b27-a9db-50680f9d36d7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.651242 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.651894 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.652180 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.652462 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.653338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] RBAC: Adding query filter params (role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=3d209aef7bbd4477a103fd4064afe530, user.id=0fe86d9a6c224cacaa25f0d29d448f20) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:49.653651 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.655102 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.660278 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 47/89] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/role_assignments?user.id=0fe86d9a6c224cacaa25f0d29d448f20&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:49.661078 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.661977 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.662640 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.662931 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.663214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.664068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=3d209aef7bbd4477a103fd4064afe530, user.id=05c2f4bd321b4a50a016143bb5412e00) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:49.664388 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.665912 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.685076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 43/90] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/role_assignments?user.id=05c2f4bd321b4a50a016143bb5412e00&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=58c34600086c469b87bfa39b74dd2607 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:49.685076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.776779 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13cd0-94ca-4f0c-90d7-3d6bbc30bdc7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.803592 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432910fa-07cb-467f-9200-63ca6987440d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:49.814641 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62397c75-e615-415e-b08b-58d0690bed91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.815350 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62397c75-e615-415e-b08b-58d0690bed91 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.815637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62397c75-e615-415e-b08b-58d0690bed91 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.815918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62397c75-e615-415e-b08b-58d0690bed91 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.816682 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62397c75-e615-415e-b08b-58d0690bed91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.818836 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62397c75-e615-415e-b08b-58d0690bed91 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.845493 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f594d46-cc7f-46bf-96b8-f0c8bc8e90ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:49.846717 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f594d46-cc7f-46bf-96b8-f0c8bc8e90ff None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:49.847100 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f594d46-cc7f-46bf-96b8-f0c8bc8e90ff None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:49.847390 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f594d46-cc7f-46bf-96b8-f0c8bc8e90ff None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:49.848938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f594d46-cc7f-46bf-96b8-f0c8bc8e90ff None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:49.849322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 48/91] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:27:49 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:27:49.850700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f594d46-cc7f-46bf-96b8-f0c8bc8e90ff None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:49.852719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 44/92] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:49.854243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62397c75-e615-415e-b08b-58d0690bed91 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:49.865325 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f594d46-cc7f-46bf-96b8-f0c8bc8e90ff None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:50.000308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62397c75-e615-415e-b08b-58d0690bed91 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:50.041888 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894dde7-2582-45e1-8874-6a7eb6a9e177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:50.043204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894dde7-2582-45e1-8874-6a7eb6a9e177 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:50.043561 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894dde7-2582-45e1-8874-6a7eb6a9e177 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:50.043843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894dde7-2582-45e1-8874-6a7eb6a9e177 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:50.045213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894dde7-2582-45e1-8874-6a7eb6a9e177 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:50.047187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894dde7-2582-45e1-8874-6a7eb6a9e177 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:50.049893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 49/93] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:50.055157 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894dde7-2582-45e1-8874-6a7eb6a9e177 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:50.072099 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f594d46-cc7f-46bf-96b8-f0c8bc8e90ff None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:50.113345 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facae99-30f1-442d-a328-9734858311eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:50.114955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facae99-30f1-442d-a328-9734858311eb None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:50.115501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facae99-30f1-442d-a328-9734858311eb None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:50.116000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facae99-30f1-442d-a328-9734858311eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:50.190106 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facae99-30f1-442d-a328-9734858311eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:50.192000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894dde7-2582-45e1-8874-6a7eb6a9e177 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:50.192560 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facae99-30f1-442d-a328-9734858311eb None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:50.233636 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:50.234310 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:50.234598 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:50.234879 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:50.235728 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] RBAC: Adding query filter params (role.id=02f09975329342eead41c01b150325c0, scope.project.id=3d209aef7bbd4477a103fd4064afe530, user.id=0fe86d9a6c224cacaa25f0d29d448f20) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:50.236044 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:50.237993 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:50.246254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 50/94] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:27:50 2022] GET /identity/v3/role_assignments?user.id=0fe86d9a6c224cacaa25f0d29d448f20&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=02f09975329342eead41c01b150325c0 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:50.309727 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 45/95] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:27:50.315733 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:50.445471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0083fc3-3ba6-4599-9f04-910114c729a9 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:50.485222 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6e8a2-5f83-4dbb-ba12-796a800ffe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:50.486121 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e8a2-5f83-4dbb-ba12-796a800ffe5f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:50.486401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e8a2-5f83-4dbb-ba12-796a800ffe5f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:50.486679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e8a2-5f83-4dbb-ba12-796a800ffe5f None admin] PATH_INFO: `/v3/projects/5874c0dfc2ac4581bb3cf5dc4a692c54` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:50.487969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6e8a2-5f83-4dbb-ba12-796a800ffe5f None admin] RBAC: Authorizing `identity:get_project(project_id=5874c0dfc2ac4581bb3cf5dc4a692c54)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:50.489569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6e8a2-5f83-4dbb-ba12-796a800ffe5f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:50.491654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 51/96] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:50 2022] GET /identity/v3/projects/5874c0dfc2ac4581bb3cf5dc4a692c54 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:50.498120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facae99-30f1-442d-a328-9734858311eb None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:50.610546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facae99-30f1-442d-a328-9734858311eb None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:50.647393 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6def70-5fa8-4a37-a2fa-d48125d78ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:50.648062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6def70-5fa8-4a37-a2fa-d48125d78ddb None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:50.648348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6def70-5fa8-4a37-a2fa-d48125d78ddb None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:50.648625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6def70-5fa8-4a37-a2fa-d48125d78ddb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:50.649566 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6def70-5fa8-4a37-a2fa-d48125d78ddb None admin] RBAC: Adding query filter params (role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=5874c0dfc2ac4581bb3cf5dc4a692c54, user.id=0fe86d9a6c224cacaa25f0d29d448f20) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:50.649883 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6def70-5fa8-4a37-a2fa-d48125d78ddb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:50.651402 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6def70-5fa8-4a37-a2fa-d48125d78ddb None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:50.657718 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 46/97] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:27:50 2022] GET /identity/v3/role_assignments?user.id=0fe86d9a6c224cacaa25f0d29d448f20&scope.project.id=5874c0dfc2ac4581bb3cf5dc4a692c54&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:52.535305 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e1521-ed3b-4c04-a28e-c029f79dd25a None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:52.536375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e1521-ed3b-4c04-a28e-c029f79dd25a None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:52.536901 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e1521-ed3b-4c04-a28e-c029f79dd25a None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:52.538807 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 52/98] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:52.544481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b196e-faf3-46f8-9e9b-a630f1d5dff4 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:52.545461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b196e-faf3-46f8-9e9b-a630f1d5dff4 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:52.545986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b196e-faf3-46f8-9e9b-a630f1d5dff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:52.569739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-a27b196e-faf3-46f8-9e9b-a630f1d5dff4 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:52.571274 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-a27b196e-faf3-46f8-9e9b-a630f1d5dff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:52.651063 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 47/99] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:52.694653 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e1521-ed3b-4c04-a28e-c029f79dd25a None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:52.804406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e1521-ed3b-4c04-a28e-c029f79dd25a None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:52.842104 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12966083-a872-492d-99a6-1efc2d2125dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:52.843336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12966083-a872-492d-99a6-1efc2d2125dd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:52.843858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12966083-a872-492d-99a6-1efc2d2125dd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:52.844348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12966083-a872-492d-99a6-1efc2d2125dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:52.856522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12966083-a872-492d-99a6-1efc2d2125dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-12966083-a872-492d-99a6-1efc2d2125dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:52.865671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:52.866660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:52.867510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:52.870313 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 53/100] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:27:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:52.878167 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b196e-faf3-46f8-9e9b-a630f1d5dff4 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:52.999635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b196e-faf3-46f8-9e9b-a630f1d5dff4 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:53.040274 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87593635-6283-485e-b273-bb11fd8aece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:53.041367 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87593635-6283-485e-b273-bb11fd8aece7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.041860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87593635-6283-485e-b273-bb11fd8aece7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.042372 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87593635-6283-485e-b273-bb11fd8aece7 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.043326 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87593635-6283-485e-b273-bb11fd8aece7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:53.043919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87593635-6283-485e-b273-bb11fd8aece7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:53.045602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87593635-6283-485e-b273-bb11fd8aece7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:53.060280 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 48/101] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:53.066165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12966083-a872-492d-99a6-1efc2d2125dd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:53.103162 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bbbba-8454-4c11-89c3-c50b443249bc None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.103766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bbbba-8454-4c11-89c3-c50b443249bc None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.104253 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bbbba-8454-4c11-89c3-c50b443249bc None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.105870 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 49/102] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:53.111407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749501f6-21e1-4f71-b701-82e7710183a7 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.111937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749501f6-21e1-4f71-b701-82e7710183a7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.112443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749501f6-21e1-4f71-b701-82e7710183a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.135159 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-749501f6-21e1-4f71-b701-82e7710183a7 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:53.136698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-749501f6-21e1-4f71-b701-82e7710183a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:53.249150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12966083-a872-492d-99a6-1efc2d2125dd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:53.258728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 50/103] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:53.260488 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bb638-17ae-4d5c-bd73-4e7ff782af90 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.260990 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bb638-17ae-4d5c-bd73-4e7ff782af90 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.261514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bb638-17ae-4d5c-bd73-4e7ff782af90 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.263678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 51/104] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:53.269384 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef365b4c-f335-42ad-9cc9-b6093e2db83f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.270270 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef365b4c-f335-42ad-9cc9-b6093e2db83f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.270770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef365b4c-f335-42ad-9cc9-b6093e2db83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.304213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-ef365b4c-f335-42ad-9cc9-b6093e2db83f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:53.305447 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-ef365b4c-f335-42ad-9cc9-b6093e2db83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:53.320074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2da4b6-7f50-4de8-bf3d-e443b062b2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:53.320892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da4b6-7f50-4de8-bf3d-e443b062b2d9 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.321213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da4b6-7f50-4de8-bf3d-e443b062b2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.321493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da4b6-7f50-4de8-bf3d-e443b062b2d9 None admin] PATH_INFO: `/v3/users/05c2f4bd321b4a50a016143bb5412e00` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.322740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2da4b6-7f50-4de8-bf3d-e443b062b2d9 None admin] RBAC: Authorizing `identity:get_user(user_id=05c2f4bd321b4a50a016143bb5412e00)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:53.324218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2da4b6-7f50-4de8-bf3d-e443b062b2d9 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:53.327986 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 54/105] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:53 2022] GET /identity/v3/users/05c2f4bd321b4a50a016143bb5412e00 => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:53.328803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2da4b6-7f50-4de8-bf3d-e443b062b2d9 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:53.433022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 52/106] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:53.433908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef365b4c-f335-42ad-9cc9-b6093e2db83f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:53.456421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2da4b6-7f50-4de8-bf3d-e443b062b2d9 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:53.503103 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447637f4-793e-41c5-9a37-e9870596e7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:53.504341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447637f4-793e-41c5-9a37-e9870596e7ed None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.504849 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447637f4-793e-41c5-9a37-e9870596e7ed None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.505354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447637f4-793e-41c5-9a37-e9870596e7ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.524045 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447637f4-793e-41c5-9a37-e9870596e7ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-447637f4-793e-41c5-9a37-e9870596e7ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:53.532745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:53.533806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:53.534844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:53.537839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 55/107] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:53.538785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447637f4-793e-41c5-9a37-e9870596e7ed None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:53.627384 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef365b4c-f335-42ad-9cc9-b6093e2db83f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:53.640334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447637f4-793e-41c5-9a37-e9870596e7ed None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:53.675137 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845ae53-aabd-402e-b699-1a210509d0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:53.676383 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845ae53-aabd-402e-b699-1a210509d0b8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.676890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845ae53-aabd-402e-b699-1a210509d0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.677409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845ae53-aabd-402e-b699-1a210509d0b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.690774 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd673e-b0f7-445b-9723-b990ca13d402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:53.691994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd673e-b0f7-445b-9723-b990ca13d402 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.692980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd673e-b0f7-445b-9723-b990ca13d402 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.693298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845ae53-aabd-402e-b699-1a210509d0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-2845ae53-aabd-402e-b699-1a210509d0b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:53.701523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:53.702451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:27:53.703375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:53.704846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd673e-b0f7-445b-9723-b990ca13d402 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.706411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd673e-b0f7-445b-9723-b990ca13d402 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:53.708627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd673e-b0f7-445b-9723-b990ca13d402 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:53.710941 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 53/108] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:53 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:53.711930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd673e-b0f7-445b-9723-b990ca13d402 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:53.713772 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 56/109] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:27:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:53.726089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845ae53-aabd-402e-b699-1a210509d0b8 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:53.841207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd673e-b0f7-445b-9723-b990ca13d402 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:53.860472 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845ae53-aabd-402e-b699-1a210509d0b8 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:53.883344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:53.884334 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.884838 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.885744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.887100 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] RBAC: Adding query filter params (name=member) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:53.887667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:53.890186 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:53.900992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba397c-e46d-435c-9581-561c428e4ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:53.901753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 54/110] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:27:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:53.902123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba397c-e46d-435c-9581-561c428e4ccd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.902606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba397c-e46d-435c-9581-561c428e4ccd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.902978 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:53.903226 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba397c-e46d-435c-9581-561c428e4ccd None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.904330 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba397c-e46d-435c-9581-561c428e4ccd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:53.904812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba397c-e46d-435c-9581-561c428e4ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:53.906772 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba397c-e46d-435c-9581-561c428e4ccd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:53.918293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 57/111] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:27:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:53.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fa648-7c82-40ef-94db-4e0d36924d84 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:53.921417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fa648-7c82-40ef-94db-4e0d36924d84 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:53.921877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fa648-7c82-40ef-94db-4e0d36924d84 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:53.928768 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 58/112] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:53.928768 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fa648-7c82-40ef-94db-4e0d36924d84 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.023193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc5c1c-075c-4270-beaa-e83dd84e9544 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.043960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fa648-7c82-40ef-94db-4e0d36924d84 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.064476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e18d5-62f1-4e87-a055-c6665666e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.065466 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e18d5-62f1-4e87-a055-c6665666e4be None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.065760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e18d5-62f1-4e87-a055-c6665666e4be None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.066039 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e18d5-62f1-4e87-a055-c6665666e4be None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530/users/05c2f4bd321b4a50a016143bb5412e00/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.068604 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e18d5-62f1-4e87-a055-c6665666e4be None admin] RBAC: Authorizing `identity:create_grant(project_id=3d209aef7bbd4477a103fd4064afe530, user_id=05c2f4bd321b4a50a016143bb5412e00, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:54.070086 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e18d5-62f1-4e87-a055-c6665666e4be None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:54.084780 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 55/113] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:27:53 2022] PUT /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530/users/05c2f4bd321b4a50a016143bb5412e00/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:27:54.093451 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e18d5-62f1-4e87-a055-c6665666e4be None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.111254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41c7d7-3fa6-489e-8686-62808c9bce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.112431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41c7d7-3fa6-489e-8686-62808c9bce2c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.112971 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41c7d7-3fa6-489e-8686-62808c9bce2c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.113367 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41c7d7-3fa6-489e-8686-62808c9bce2c None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.115649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41c7d7-3fa6-489e-8686-62808c9bce2c None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:54.117877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41c7d7-3fa6-489e-8686-62808c9bce2c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:54.119966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 59/114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:53 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:54.121924 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be8912-b5fd-463d-aca7-8bff37a2be90 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.122372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be8912-b5fd-463d-aca7-8bff37a2be90 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.122806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be8912-b5fd-463d-aca7-8bff37a2be90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.146205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-e5be8912-b5fd-463d-aca7-8bff37a2be90 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:54.147945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-e5be8912-b5fd-463d-aca7-8bff37a2be90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:54.203093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e18d5-62f1-4e87-a055-c6665666e4be None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.222163 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 60/115] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:54.226893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38cb5e-8ba9-4b81-b728-2ca96baa8d16 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.227364 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38cb5e-8ba9-4b81-b728-2ca96baa8d16 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.227716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38cb5e-8ba9-4b81-b728-2ca96baa8d16 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.230089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 61/116] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:54.231619 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91756b1-a665-4203-b84a-c95b1f395039 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.232002 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91756b1-a665-4203-b84a-c95b1f395039 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.232857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91756b1-a665-4203-b84a-c95b1f395039 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.234414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 62/117] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:54.237882 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91756b1-a665-4203-b84a-c95b1f395039 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.245915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a94e0-076f-47ea-bf67-9e79c8574257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.247180 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a94e0-076f-47ea-bf67-9e79c8574257 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.247686 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a94e0-076f-47ea-bf67-9e79c8574257 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.248175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a94e0-076f-47ea-bf67-9e79c8574257 None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.249735 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a94e0-076f-47ea-bf67-9e79c8574257 None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:54.251628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a94e0-076f-47ea-bf67-9e79c8574257 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:54.254005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 56/118] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:54 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:54.256260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf997d8b-d047-40b1-8b5e-c3978b9900e2 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.256557 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf997d8b-d047-40b1-8b5e-c3978b9900e2 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.256839 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf997d8b-d047-40b1-8b5e-c3978b9900e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.277987 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-bf997d8b-d047-40b1-8b5e-c3978b9900e2 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:54.279645 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-bf997d8b-d047-40b1-8b5e-c3978b9900e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:54.352894 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91756b1-a665-4203-b84a-c95b1f395039 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.354117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 57/119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:54.355577 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263775d-f693-4db3-82b9-e3347d62ddd0 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.355873 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263775d-f693-4db3-82b9-e3347d62ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.356162 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263775d-f693-4db3-82b9-e3347d62ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.375169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-7263775d-f693-4db3-82b9-e3347d62ddd0 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:54.376827 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-7263775d-f693-4db3-82b9-e3347d62ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:54.396028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aded030-46d3-42d7-ab8d-601214e80d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.396922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded030-46d3-42d7-ab8d-601214e80d9e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.397235 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded030-46d3-42d7-ab8d-601214e80d9e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.397514 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded030-46d3-42d7-ab8d-601214e80d9e None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.399289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aded030-46d3-42d7-ab8d-601214e80d9e None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:54.401367 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aded030-46d3-42d7-ab8d-601214e80d9e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:54.407466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 63/120] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:54 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:54.407466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aded030-46d3-42d7-ab8d-601214e80d9e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.451643 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 58/121] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:54.452513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263775d-f693-4db3-82b9-e3347d62ddd0 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.510477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aded030-46d3-42d7-ab8d-601214e80d9e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.544602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08eb90e-87a4-4319-b2ac-29e719430411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.545476 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eb90e-87a4-4319-b2ac-29e719430411 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.545767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eb90e-87a4-4319-b2ac-29e719430411 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.546041 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08eb90e-87a4-4319-b2ac-29e719430411 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.547311 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08eb90e-87a4-4319-b2ac-29e719430411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:54.548834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08eb90e-87a4-4319-b2ac-29e719430411 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:54.552613 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263775d-f693-4db3-82b9-e3347d62ddd0 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.555698 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 64/122] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:54.556410 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08eb90e-87a4-4319-b2ac-29e719430411 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.586539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a225ce2-80aa-488b-8c5b-9b1092f2be3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.587410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a225ce2-80aa-488b-8c5b-9b1092f2be3e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.587703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a225ce2-80aa-488b-8c5b-9b1092f2be3e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.587981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a225ce2-80aa-488b-8c5b-9b1092f2be3e None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.589223 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a225ce2-80aa-488b-8c5b-9b1092f2be3e None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:54.590698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a225ce2-80aa-488b-8c5b-9b1092f2be3e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:54.592572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 59/123] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:54 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:54.593283 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a225ce2-80aa-488b-8c5b-9b1092f2be3e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.659029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08eb90e-87a4-4319-b2ac-29e719430411 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.693923 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a225ce2-80aa-488b-8c5b-9b1092f2be3e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.694457 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bb6e1-b432-4e12-944f-8fa59f4a01d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.694828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb6e1-b432-4e12-944f-8fa59f4a01d6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.695118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb6e1-b432-4e12-944f-8fa59f4a01d6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.695402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb6e1-b432-4e12-944f-8fa59f4a01d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.706021 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bb6e1-b432-4e12-944f-8fa59f4a01d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-177bb6e1-b432-4e12-944f-8fa59f4a01d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:54.713922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:54.714896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:54.715803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 65/124] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:54.716543 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bb6e1-b432-4e12-944f-8fa59f4a01d6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.732052 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ad00d-a5ad-4a0d-9ae2-279dc3175616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.733563 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ad00d-a5ad-4a0d-9ae2-279dc3175616 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.734074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ad00d-a5ad-4a0d-9ae2-279dc3175616 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.734579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ad00d-a5ad-4a0d-9ae2-279dc3175616 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530/users/0fe86d9a6c224cacaa25f0d29d448f20/roles/251b2e26a5834411bf902ba50b5ec81f` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.736767 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ad00d-a5ad-4a0d-9ae2-279dc3175616 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d209aef7bbd4477a103fd4064afe530, user_id=0fe86d9a6c224cacaa25f0d29d448f20, role_id=251b2e26a5834411bf902ba50b5ec81f)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:54.738475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ad00d-a5ad-4a0d-9ae2-279dc3175616 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:54.753687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 60/125] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:27:54 2022] PUT /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530/users/0fe86d9a6c224cacaa25f0d29d448f20/roles/251b2e26a5834411bf902ba50b5ec81f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:27:54.754482 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ad00d-a5ad-4a0d-9ae2-279dc3175616 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.858248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bb6e1-b432-4e12-944f-8fa59f4a01d6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.873731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ad00d-a5ad-4a0d-9ae2-279dc3175616 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:54.907360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b73ff-1d76-4138-8c74-68f059bb48ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.908720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b73ff-1d76-4138-8c74-68f059bb48ab None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.909362 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b73ff-1d76-4138-8c74-68f059bb48ab None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.909986 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b73ff-1d76-4138-8c74-68f059bb48ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.911762 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0252b-3ca0-4674-bdf4-f1729c992a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:54.912599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0252b-3ca0-4674-bdf4-f1729c992a6d None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:54.912969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0252b-3ca0-4674-bdf4-f1729c992a6d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:54.913359 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0252b-3ca0-4674-bdf4-f1729c992a6d None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:54.914271 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0252b-3ca0-4674-bdf4-f1729c992a6d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:54.915940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0252b-3ca0-4674-bdf4-f1729c992a6d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:54.924937 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b73ff-1d76-4138-8c74-68f059bb48ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-4d5b73ff-1d76-4138-8c74-68f059bb48ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:54.935826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:54.937540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:54.938431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 66/126] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:54.943718 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b73ff-1d76-4138-8c74-68f059bb48ab None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:54.953156 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 61/127] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:27:54 2022] POST /identity/v3/projects => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:27:54.954903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0252b-3ca0-4674-bdf4-f1729c992a6d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:55.082929 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b73ff-1d76-4138-8c74-68f059bb48ab None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:55.109061 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0252b-3ca0-4674-bdf4-f1729c992a6d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:55.124182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e8001-cdc3-4b40-9882-78b0511b1ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:55.125245 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e8001-cdc3-4b40-9882-78b0511b1ee7 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:55.125658 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e8001-cdc3-4b40-9882-78b0511b1ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:55.126013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e8001-cdc3-4b40-9882-78b0511b1ee7 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530/users/0fe86d9a6c224cacaa25f0d29d448f20/roles/02f09975329342eead41c01b150325c0` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:55.128552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e8001-cdc3-4b40-9882-78b0511b1ee7 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d209aef7bbd4477a103fd4064afe530, user_id=0fe86d9a6c224cacaa25f0d29d448f20, role_id=02f09975329342eead41c01b150325c0)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:55.130323 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e8001-cdc3-4b40-9882-78b0511b1ee7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:55.146903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:55.147642 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:55.147761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 67/128] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:27:54 2022] PUT /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530/users/0fe86d9a6c224cacaa25f0d29d448f20/roles/02f09975329342eead41c01b150325c0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:27:55.147960 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:55.148220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:55.148542 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e8001-cdc3-4b40-9882-78b0511b1ee7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:55.148994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] RBAC: Adding query filter params (name=reader) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:55.153470 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:55.158106 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:55.176910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 62/129] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:27:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:55.186865 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:55.378972 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e8001-cdc3-4b40-9882-78b0511b1ee7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:55.403426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bed8b-3731-4eec-a7f9-e24bd8cea406 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:55.432040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:55.432454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:55.432748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:55.433031 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:55.433830 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:55.434171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:55.435568 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:55.446364 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9fc56-9fff-4164-9fc9-9dec2831e237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:55.446688 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 68/130] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:27:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:55.447745 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9fc56-9fff-4164-9fc9-9dec2831e237 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:55.448038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9fc56-9fff-4164-9fc9-9dec2831e237 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:55.448320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9fc56-9fff-4164-9fc9-9dec2831e237 None admin] PATH_INFO: `/v3/users/6b22696ebeea4ff4b2eb37529ad3e429` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:55.452656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:55.465872 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9fc56-9fff-4164-9fc9-9dec2831e237 None admin] RBAC: Authorizing `identity:get_user(user_id=6b22696ebeea4ff4b2eb37529ad3e429)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:55.467407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9fc56-9fff-4164-9fc9-9dec2831e237 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:55.469926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 63/131] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:55 2022] GET /identity/v3/users/6b22696ebeea4ff4b2eb37529ad3e429 => generated 289 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:55.477192 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9fc56-9fff-4164-9fc9-9dec2831e237 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:55.583270 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c77c0a-4189-4615-b77e-7bc48bd35097 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:55.598821 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9fc56-9fff-4164-9fc9-9dec2831e237 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:55.617815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133209-0214-42e6-b0fb-7dee47638b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:55.618760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133209-0214-42e6-b0fb-7dee47638b62 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:55.619054 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133209-0214-42e6-b0fb-7dee47638b62 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:55.619337 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133209-0214-42e6-b0fb-7dee47638b62 None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:55.620592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1133209-0214-42e6-b0fb-7dee47638b62 None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:55.622159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1133209-0214-42e6-b0fb-7dee47638b62 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:55.623845 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 69/132] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:55 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:55.629788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133209-0214-42e6-b0fb-7dee47638b62 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:55.637236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb8540-c190-4b2f-97c1-04d12a796b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:55.638314 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8540-c190-4b2f-97c1-04d12a796b04 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:55.638834 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8540-c190-4b2f-97c1-04d12a796b04 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:55.639340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8540-c190-4b2f-97c1-04d12a796b04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:55.707822 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb8540-c190-4b2f-97c1-04d12a796b04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:55.710651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb8540-c190-4b2f-97c1-04d12a796b04 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:55.732743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133209-0214-42e6-b0fb-7dee47638b62 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:55.767822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9d973-c0d3-449c-a9b9-dce05b0a5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:55.768709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9d973-c0d3-449c-a9b9-dce05b0a5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:55.769020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9d973-c0d3-449c-a9b9-dce05b0a5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:55.769331 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9d973-c0d3-449c-a9b9-dce05b0a5cf6 None admin] PATH_INFO: `/v3/projects/5874c0dfc2ac4581bb3cf5dc4a692c54` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:55.770583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9d973-c0d3-449c-a9b9-dce05b0a5cf6 None admin] RBAC: Authorizing `identity:get_project(project_id=5874c0dfc2ac4581bb3cf5dc4a692c54)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:55.772067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9d973-c0d3-449c-a9b9-dce05b0a5cf6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:55.773961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 70/133] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:55 2022] GET /identity/v3/projects/5874c0dfc2ac4581bb3cf5dc4a692c54 => generated 312 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:55.779728 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9d973-c0d3-449c-a9b9-dce05b0a5cf6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:55.815877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 64/134] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:27:55.826183 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb8540-c190-4b2f-97c1-04d12a796b04 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:55.971072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9d973-c0d3-449c-a9b9-dce05b0a5cf6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:55.986926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb8540-c190-4b2f-97c1-04d12a796b04 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:56.036531 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568034f8-197c-4fb6-86fe-45d80afdc2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:56.037819 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568034f8-197c-4fb6-86fe-45d80afdc2d9 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:56.038334 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568034f8-197c-4fb6-86fe-45d80afdc2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:56.038858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568034f8-197c-4fb6-86fe-45d80afdc2d9 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:56.040310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568034f8-197c-4fb6-86fe-45d80afdc2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:56.042362 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568034f8-197c-4fb6-86fe-45d80afdc2d9 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:56.044318 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 65/135] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:55 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:56.050261 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568034f8-197c-4fb6-86fe-45d80afdc2d9 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:56.050757 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f05b5-937c-4c5b-b103-3a4eaf4cc167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:56.052226 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f05b5-937c-4c5b-b103-3a4eaf4cc167 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:56.052518 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f05b5-937c-4c5b-b103-3a4eaf4cc167 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:56.052804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f05b5-937c-4c5b-b103-3a4eaf4cc167 None admin] PATH_INFO: `/v3/projects/5874c0dfc2ac4581bb3cf5dc4a692c54/users/0fe86d9a6c224cacaa25f0d29d448f20/roles/251b2e26a5834411bf902ba50b5ec81f` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:56.058425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f05b5-937c-4c5b-b103-3a4eaf4cc167 None admin] RBAC: Authorizing `identity:create_grant(project_id=5874c0dfc2ac4581bb3cf5dc4a692c54, user_id=0fe86d9a6c224cacaa25f0d29d448f20, role_id=251b2e26a5834411bf902ba50b5ec81f)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:56.059918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f05b5-937c-4c5b-b103-3a4eaf4cc167 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:56.105529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 71/136] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:27:55 2022] PUT /identity/v3/projects/5874c0dfc2ac4581bb3cf5dc4a692c54/users/0fe86d9a6c224cacaa25f0d29d448f20/roles/251b2e26a5834411bf902ba50b5ec81f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:27:56.190025 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568034f8-197c-4fb6-86fe-45d80afdc2d9 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:56.226115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52feed1-e3b7-4d0e-b190-e2138e064900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:56.226734 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52feed1-e3b7-4d0e-b190-e2138e064900 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:56.227023 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52feed1-e3b7-4d0e-b190-e2138e064900 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:56.227305 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52feed1-e3b7-4d0e-b190-e2138e064900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:56.228174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52feed1-e3b7-4d0e-b190-e2138e064900 None admin] RBAC: Adding query filter params (role.id=6d053d567af44b6685295deb347f550e, scope.project.id=3d209aef7bbd4477a103fd4064afe530, user.id=6b22696ebeea4ff4b2eb37529ad3e429) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:56.228495 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52feed1-e3b7-4d0e-b190-e2138e064900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:56.229997 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52feed1-e3b7-4d0e-b190-e2138e064900 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:56.236405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 66/137] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:27:56 2022] GET /identity/v3/role_assignments?user.id=6b22696ebeea4ff4b2eb37529ad3e429&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=6d053d567af44b6685295deb347f550e => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:56.889997 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa082e-74b2-4aa0-91bf-e41f8c927be9 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:56.890544 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa082e-74b2-4aa0-91bf-e41f8c927be9 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:56.890641 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa082e-74b2-4aa0-91bf-e41f8c927be9 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:56.892392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 72/138] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:56.910832 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fdccd-06ac-4d73-a4bc-9eebd2f9442f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:56.911337 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fdccd-06ac-4d73-a4bc-9eebd2f9442f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:56.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fdccd-06ac-4d73-a4bc-9eebd2f9442f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:56.928488 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-543fdccd-06ac-4d73-a4bc-9eebd2f9442f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:56.929725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-543fdccd-06ac-4d73-a4bc-9eebd2f9442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:56.998416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 67/139] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:57.027938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa082e-74b2-4aa0-91bf-e41f8c927be9 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:57.157861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa082e-74b2-4aa0-91bf-e41f8c927be9 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:57.198683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8b045-ab0a-4851-9150-55f7f39698c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:57.199490 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8b045-ab0a-4851-9150-55f7f39698c4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:57.199783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8b045-ab0a-4851-9150-55f7f39698c4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:57.200066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8b045-ab0a-4851-9150-55f7f39698c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:57.213857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8b045-ab0a-4851-9150-55f7f39698c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-36c8b045-ab0a-4851-9150-55f7f39698c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:57.221765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:57.222709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:27:57.223668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:57.226856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 73/140] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:27:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:27:57.233274 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fdccd-06ac-4d73-a4bc-9eebd2f9442f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:57.396820 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fdccd-06ac-4d73-a4bc-9eebd2f9442f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:57.442644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:57.443258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:57.443550 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:57.443839 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:57.444615 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:57.444936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:57.446379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:57.460674 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 68/141] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:57.466200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8b045-ab0a-4851-9150-55f7f39698c4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:57.631778 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8b045-ab0a-4851-9150-55f7f39698c4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:57.673903 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e652323-7eb2-471f-93c9-877baf7a5bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:57.674791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e652323-7eb2-471f-93c9-877baf7a5bae None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:57.675131 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e652323-7eb2-471f-93c9-877baf7a5bae None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:57.675416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e652323-7eb2-471f-93c9-877baf7a5bae None admin] PATH_INFO: `/v3/users/05c2f4bd321b4a50a016143bb5412e00` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:57.680394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e652323-7eb2-471f-93c9-877baf7a5bae None admin] RBAC: Authorizing `identity:get_user(user_id=05c2f4bd321b4a50a016143bb5412e00)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:57.681951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e652323-7eb2-471f-93c9-877baf7a5bae None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:57.683613 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 74/142] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:57 2022] GET /identity/v3/users/05c2f4bd321b4a50a016143bb5412e00 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:57.689126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:57.828823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc7206-b281-41b8-8781-4b44b68203b0 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:57.875276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea674457-bfb2-442c-8848-13220f958d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:57.877935 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea674457-bfb2-442c-8848-13220f958d1b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:57.879066 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea674457-bfb2-442c-8848-13220f958d1b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:57.879066 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea674457-bfb2-442c-8848-13220f958d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:57.944776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6ea6-6af8-4346-9431-c65ad7d0b5bd None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:57.945321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6ea6-6af8-4346-9431-c65ad7d0b5bd None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:57.945422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6ea6-6af8-4346-9431-c65ad7d0b5bd None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:57.947976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 75/143] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:57.951781 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4f82-aefc-4f51-b81a-5973b5cc07f6 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:57.952085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4f82-aefc-4f51-b81a-5973b5cc07f6 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:57.952370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4f82-aefc-4f51-b81a-5973b5cc07f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:57.961879 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea674457-bfb2-442c-8848-13220f958d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:57.963373 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea674457-bfb2-442c-8848-13220f958d1b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:57.979126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-662e4f82-aefc-4f51-b81a-5973b5cc07f6 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:57.980338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-662e4f82-aefc-4f51-b81a-5973b5cc07f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:58.062752 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 69/144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:27:58.068060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea674457-bfb2-442c-8848-13220f958d1b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:58.103107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 76/145] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:58.138312 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4f82-aefc-4f51-b81a-5973b5cc07f6 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:58.176429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea674457-bfb2-442c-8848-13220f958d1b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:58.211312 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d372a4-980b-44cf-b8d6-5996ffe909bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:58.212182 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d372a4-980b-44cf-b8d6-5996ffe909bd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:58.212525 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d372a4-980b-44cf-b8d6-5996ffe909bd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:58.212826 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d372a4-980b-44cf-b8d6-5996ffe909bd None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:58.214093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d372a4-980b-44cf-b8d6-5996ffe909bd None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:58.215558 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d372a4-980b-44cf-b8d6-5996ffe909bd None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:58.217341 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 70/146] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:58 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:58.223854 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d372a4-980b-44cf-b8d6-5996ffe909bd None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:58.328609 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d372a4-980b-44cf-b8d6-5996ffe909bd None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:58.338437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4f82-aefc-4f51-b81a-5973b5cc07f6 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:58.363991 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:58.364589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:58.364883 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:58.365206 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:58.366086 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=3d209aef7bbd4477a103fd4064afe530, user.id=05c2f4bd321b4a50a016143bb5412e00) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:58.366409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:58.367884 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:58.374849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 71/147] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:27:58 2022] GET /identity/v3/role_assignments?user.id=05c2f4bd321b4a50a016143bb5412e00&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=58c34600086c469b87bfa39b74dd2607 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:58.403026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e1b53-b415-47c5-8267-3aa3cfc57304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:58.403880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e1b53-b415-47c5-8267-3aa3cfc57304 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:58.404172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e1b53-b415-47c5-8267-3aa3cfc57304 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:58.404457 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e1b53-b415-47c5-8267-3aa3cfc57304 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:58.427213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e1b53-b415-47c5-8267-3aa3cfc57304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-f49e1b53-b415-47c5-8267-3aa3cfc57304 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:58.435293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:58.437750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:27:58.439797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:58.441489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 77/148] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:27:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:27:58.449189 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:58.618245 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c38b50-3916-4d85-bd9c-295325808a6c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:58.648085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06508a-61fe-470e-836f-c3e21d3f61bb None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:58.648443 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06508a-61fe-470e-836f-c3e21d3f61bb None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:58.648660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06508a-61fe-470e-836f-c3e21d3f61bb None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:58.650106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 78/149] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:27:58.656837 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05836e-466e-4e54-9d60-5362f72c250c None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:58.657159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05836e-466e-4e54-9d60-5362f72c250c None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:58.657454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05836e-466e-4e54-9d60-5362f72c250c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:58.659692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:58.660406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:58.660700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:58.660984 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:58.661808 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] RBAC: Adding query filter params (name=member) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:58.662121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:58.663532 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:58.672894 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 72/150] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:27:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:58.681508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:58.685105 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-4a05836e-466e-4e54-9d60-5362f72c250c None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:58.686351 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-4a05836e-466e-4e54-9d60-5362f72c250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:58.757135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 79/151] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:27:58.821383 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05836e-466e-4e54-9d60-5362f72c250c None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:58.825823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7a8cf-1a17-4b0b-9b2a-1acd7e44aa83 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:58.889633 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64be25-1c7b-4a7f-a73d-b19266ec6580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:58.890458 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64be25-1c7b-4a7f-a73d-b19266ec6580 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:58.891906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64be25-1c7b-4a7f-a73d-b19266ec6580 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:58.892205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64be25-1c7b-4a7f-a73d-b19266ec6580 None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:58.898338 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64be25-1c7b-4a7f-a73d-b19266ec6580 None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:58.899908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64be25-1c7b-4a7f-a73d-b19266ec6580 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:58.903980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 73/152] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:58 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:27:58.914204 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64be25-1c7b-4a7f-a73d-b19266ec6580 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:58.998042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05836e-466e-4e54-9d60-5362f72c250c None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.028366 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64be25-1c7b-4a7f-a73d-b19266ec6580 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.054381 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc8fcb-d532-448a-95f2-639cfc25e2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:59.055217 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8fcb-d532-448a-95f2-639cfc25e2be None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.055500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8fcb-d532-448a-95f2-639cfc25e2be None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.055779 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8fcb-d532-448a-95f2-639cfc25e2be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.075024 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc8fcb-d532-448a-95f2-639cfc25e2be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-eebc8fcb-d532-448a-95f2-639cfc25e2be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:27:59.083896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:27:59.088700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:27:59.090782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:27:59.092340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6e981-7a5f-43a0-b8b0-ccbb949430f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:59.092340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e981-7a5f-43a0-b8b0-ccbb949430f3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.092340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e981-7a5f-43a0-b8b0-ccbb949430f3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.092340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6e981-7a5f-43a0-b8b0-ccbb949430f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.092938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 80/153] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:27:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:27:59.100738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc8fcb-d532-448a-95f2-639cfc25e2be None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:59.213997 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc8fcb-d532-448a-95f2-639cfc25e2be None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.226939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6e981-7a5f-43a0-b8b0-ccbb949430f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:59.228765 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6e981-7a5f-43a0-b8b0-ccbb949430f3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:59.249158 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753ccf9-efb2-41c5-8bf6-e363a479da0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:59.249862 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753ccf9-efb2-41c5-8bf6-e363a479da0a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.250150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753ccf9-efb2-41c5-8bf6-e363a479da0a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.250431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753ccf9-efb2-41c5-8bf6-e363a479da0a None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.251233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753ccf9-efb2-41c5-8bf6-e363a479da0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:59.251551 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753ccf9-efb2-41c5-8bf6-e363a479da0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:59.253074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753ccf9-efb2-41c5-8bf6-e363a479da0a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:59.263596 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 81/154] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:27:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:59.265073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65dd2f-4721-426e-8755-f427546ef383 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.265365 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65dd2f-4721-426e-8755-f427546ef383 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.265653 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65dd2f-4721-426e-8755-f427546ef383 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.267065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 82/155] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:59.272609 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65dd2f-4721-426e-8755-f427546ef383 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:59.354326 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 74/156] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:27:59.355659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d726bf-6e82-4d3f-9ed4-fede35e21768 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.355960 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d726bf-6e82-4d3f-9ed4-fede35e21768 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.356251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d726bf-6e82-4d3f-9ed4-fede35e21768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.377801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-d1d726bf-6e82-4d3f-9ed4-fede35e21768 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:59.379529 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-d1d726bf-6e82-4d3f-9ed4-fede35e21768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:59.384407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65dd2f-4721-426e-8755-f427546ef383 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.420230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c54e03-9349-410f-84ce-7198e5173363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:59.421493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c54e03-9349-410f-84ce-7198e5173363 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.422002 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c54e03-9349-410f-84ce-7198e5173363 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.422501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c54e03-9349-410f-84ce-7198e5173363 None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.423937 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c54e03-9349-410f-84ce-7198e5173363 None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:59.425635 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c54e03-9349-410f-84ce-7198e5173363 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:59.427541 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 83/157] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:27:59 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:59.428513 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c54e03-9349-410f-84ce-7198e5173363 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:59.452604 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 75/158] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:59.453505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d726bf-6e82-4d3f-9ed4-fede35e21768 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:59.539470 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c54e03-9349-410f-84ce-7198e5173363 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.554689 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d726bf-6e82-4d3f-9ed4-fede35e21768 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.573618 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b176a1e-8dd3-4491-94c0-2dcc7570159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:59.574483 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b176a1e-8dd3-4491-94c0-2dcc7570159a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.574769 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b176a1e-8dd3-4491-94c0-2dcc7570159a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.575049 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b176a1e-8dd3-4491-94c0-2dcc7570159a None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.576263 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b176a1e-8dd3-4491-94c0-2dcc7570159a None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:59.577773 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b176a1e-8dd3-4491-94c0-2dcc7570159a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:59.579651 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 84/159] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:59 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:59.580665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34c6c5-2c39-475a-8f88-ed44fc8af3e8 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.580951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34c6c5-2c39-475a-8f88-ed44fc8af3e8 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.581253 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34c6c5-2c39-475a-8f88-ed44fc8af3e8 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.585461 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 85/160] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:27:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:27:59.585461 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34c6c5-2c39-475a-8f88-ed44fc8af3e8 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:59.594872 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0bdd-e000-4c2b-aa95-648bebee76cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:59.595540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0bdd-e000-4c2b-aa95-648bebee76cb None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.595829 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0bdd-e000-4c2b-aa95-648bebee76cb None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.596110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0bdd-e000-4c2b-aa95-648bebee76cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.670459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0bdd-e000-4c2b-aa95-648bebee76cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:59.671983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0bdd-e000-4c2b-aa95-648bebee76cb None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:59.688239 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34c6c5-2c39-475a-8f88-ed44fc8af3e8 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.722267 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7d102-a6a7-4542-abd1-82d3d868f144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:59.723202 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7d102-a6a7-4542-abd1-82d3d868f144 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.723485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7d102-a6a7-4542-abd1-82d3d868f144 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.723762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7d102-a6a7-4542-abd1-82d3d868f144 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.725070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7d102-a6a7-4542-abd1-82d3d868f144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:59.726573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7d102-a6a7-4542-abd1-82d3d868f144 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:59.728507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 86/161] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:27:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:59.729136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7d102-a6a7-4542-abd1-82d3d868f144 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:59.776892 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 76/162] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:27:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:27:59.778340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adc59c-e9c5-4e10-a6e8-63ec6cf17a58 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.778634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adc59c-e9c5-4e10-a6e8-63ec6cf17a58 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.778917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88adc59c-e9c5-4e10-a6e8-63ec6cf17a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.794756 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-88adc59c-e9c5-4e10-a6e8-63ec6cf17a58 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:27:59.795909 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-88adc59c-e9c5-4e10-a6e8-63ec6cf17a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:27:59.830760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7d102-a6a7-4542-abd1-82d3d868f144 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.859635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 77/163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:27:59.860392 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adc59c-e9c5-4e10-a6e8-63ec6cf17a58 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:59.868320 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:27:59.869092 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:27:59.869383 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:27:59.869661 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:27:59.870508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] RBAC: Adding query filter params (role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=3d209aef7bbd4477a103fd4064afe530, user.id=0fe86d9a6c224cacaa25f0d29d448f20) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:27:59.870838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:27:59.872284 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:27:59.878299 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 87/164] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:27:59 2022] GET /identity/v3/role_assignments?user.id=0fe86d9a6c224cacaa25f0d29d448f20&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:27:59.879036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:27:59.971653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88adc59c-e9c5-4e10-a6e8-63ec6cf17a58 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:27:59.983382 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0e4c4-f945-46b9-bf79-3ec10416ed4e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.009567 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099644c7-8806-4688-ae12-e60eb5f22776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.010263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099644c7-8806-4688-ae12-e60eb5f22776 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.010549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099644c7-8806-4688-ae12-e60eb5f22776 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.010828 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099644c7-8806-4688-ae12-e60eb5f22776 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.011565 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099644c7-8806-4688-ae12-e60eb5f22776 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:00.013123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099644c7-8806-4688-ae12-e60eb5f22776 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:00.020453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f90d43-aadc-49b4-83ba-deba8189b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.021556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f90d43-aadc-49b4-83ba-deba8189b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.021671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f90d43-aadc-49b4-83ba-deba8189b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.021987 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f90d43-aadc-49b4-83ba-deba8189b0b0 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.023608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f90d43-aadc-49b4-83ba-deba8189b0b0 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:00.025883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f90d43-aadc-49b4-83ba-deba8189b0b0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:00.028025 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 88/165] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:27:59 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:00.030650 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa20b9-af7d-46f3-8adf-71db4432b3ca None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.031285 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa20b9-af7d-46f3-8adf-71db4432b3ca None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.031944 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa20b9-af7d-46f3-8adf-71db4432b3ca None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.034256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 89/166] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:00.035897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa20b9-af7d-46f3-8adf-71db4432b3ca None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.039192 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 78/167] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:27:59 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:00.040410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099644c7-8806-4688-ae12-e60eb5f22776 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.160156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa20b9-af7d-46f3-8adf-71db4432b3ca None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.171082 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099644c7-8806-4688-ae12-e60eb5f22776 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.201985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c330-a3e2-46d5-b241-31192bb542bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.202994 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c330-a3e2-46d5-b241-31192bb542bb None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.203434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c330-a3e2-46d5-b241-31192bb542bb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.203821 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c330-a3e2-46d5-b241-31192bb542bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.210538 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c4add-f65e-4ae6-888c-dd9d2005d3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.211218 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c4add-f65e-4ae6-888c-dd9d2005d3f4 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.211507 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c4add-f65e-4ae6-888c-dd9d2005d3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.211782 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c4add-f65e-4ae6-888c-dd9d2005d3f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.212642 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c4add-f65e-4ae6-888c-dd9d2005d3f4 None admin] RBAC: Adding query filter params (role.id=02f09975329342eead41c01b150325c0, scope.project.id=3d209aef7bbd4477a103fd4064afe530, user.id=0fe86d9a6c224cacaa25f0d29d448f20) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:00.212963 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c4add-f65e-4ae6-888c-dd9d2005d3f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:00.214476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c4add-f65e-4ae6-888c-dd9d2005d3f4 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:00.218731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170c330-a3e2-46d5-b241-31192bb542bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:00.221073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 79/168] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/role_assignments?user.id=0fe86d9a6c224cacaa25f0d29d448f20&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=02f09975329342eead41c01b150325c0 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:00.222525 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11029812-8ec6-494a-87d3-febbaf8f17a1 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.222877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11029812-8ec6-494a-87d3-febbaf8f17a1 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.223667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11029812-8ec6-494a-87d3-febbaf8f17a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-6170c330-a3e2-46d5-b241-31192bb542bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:00.228145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:00.229309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:00.230201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 90/169] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:00.236382 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c330-a3e2-46d5-b241-31192bb542bb None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.245506 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-11029812-8ec6-494a-87d3-febbaf8f17a1 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:00.247641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-11029812-8ec6-494a-87d3-febbaf8f17a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:00.324268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 80/170] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:00.348794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11029812-8ec6-494a-87d3-febbaf8f17a1 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.380908 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c330-a3e2-46d5-b241-31192bb542bb None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.420419 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.421189 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.421577 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.421953 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.423777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:00.423777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:00.424668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:00.435198 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 91/171] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:00.444367 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.479779 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11029812-8ec6-494a-87d3-febbaf8f17a1 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.522260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4df74-f57e-4f49-9184-83f0db89c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.523415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4df74-f57e-4f49-9184-83f0db89c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.523916 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4df74-f57e-4f49-9184-83f0db89c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.524396 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4df74-f57e-4f49-9184-83f0db89c3b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.539630 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4df74-f57e-4f49-9184-83f0db89c3b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-def4df74-f57e-4f49-9184-83f0db89c3b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:00.547622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:00.548637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:00.549635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 81/172] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:00.556727 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4df74-f57e-4f49-9184-83f0db89c3b6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.591528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d57dd8-6e1a-4e5e-bcc5-dd204b008a65 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.639664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5114aa-6a64-4737-b99e-1375fa3c508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.640904 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5114aa-6a64-4737-b99e-1375fa3c508a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.641452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5114aa-6a64-4737-b99e-1375fa3c508a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.641983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5114aa-6a64-4737-b99e-1375fa3c508a None admin] PATH_INFO: `/v3/users/6b22696ebeea4ff4b2eb37529ad3e429` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.643480 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5114aa-6a64-4737-b99e-1375fa3c508a None admin] RBAC: Authorizing `identity:get_user(user_id=6b22696ebeea4ff4b2eb37529ad3e429)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:00.645938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5114aa-6a64-4737-b99e-1375fa3c508a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:00.647863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 92/173] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/users/6b22696ebeea4ff4b2eb37529ad3e429 => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:00.653511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5114aa-6a64-4737-b99e-1375fa3c508a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.679544 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4df74-f57e-4f49-9184-83f0db89c3b6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.723082 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.723768 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.724053 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.724333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.725447 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] RBAC: Adding query filter params (name=member) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:00.725447 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:00.726866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:00.736740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 82/174] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:00.742268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.779303 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5114aa-6a64-4737-b99e-1375fa3c508a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.818060 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698038-94c3-4569-8ce7-bbdcff1933b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.819316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698038-94c3-4569-8ce7-bbdcff1933b7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.819816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698038-94c3-4569-8ce7-bbdcff1933b7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.820461 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698038-94c3-4569-8ce7-bbdcff1933b7 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.822019 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c698038-94c3-4569-8ce7-bbdcff1933b7 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:00.823693 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c698038-94c3-4569-8ce7-bbdcff1933b7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:00.825703 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 93/175] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:00.831388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698038-94c3-4569-8ce7-bbdcff1933b7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.871669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512500c3-c9c5-469d-9188-bce09a95bcba None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:00.916088 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3d11a-3784-4ee0-808b-4790bcd1d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:00.917293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3d11a-3784-4ee0-808b-4790bcd1d1ab None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:00.917587 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3d11a-3784-4ee0-808b-4790bcd1d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:00.917871 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3d11a-3784-4ee0-808b-4790bcd1d1ab None admin] PATH_INFO: `/v3/users/0fe86d9a6c224cacaa25f0d29d448f20` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:00.919127 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3d11a-3784-4ee0-808b-4790bcd1d1ab None admin] RBAC: Authorizing `identity:get_user(user_id=0fe86d9a6c224cacaa25f0d29d448f20)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:00.921216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3d11a-3784-4ee0-808b-4790bcd1d1ab None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:00.923649 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 83/176] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/users/0fe86d9a6c224cacaa25f0d29d448f20 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:00.928897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3d11a-3784-4ee0-808b-4790bcd1d1ab None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:00.951179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698038-94c3-4569-8ce7-bbdcff1933b7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:01.013776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39452c3-2afa-41d2-9a3f-1952a673c3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:01.014983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39452c3-2afa-41d2-9a3f-1952a673c3a1 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:01.015441 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39452c3-2afa-41d2-9a3f-1952a673c3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:01.015891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39452c3-2afa-41d2-9a3f-1952a673c3a1 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530/users/6b22696ebeea4ff4b2eb37529ad3e429/roles/6d053d567af44b6685295deb347f550e` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:01.018625 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39452c3-2afa-41d2-9a3f-1952a673c3a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d209aef7bbd4477a103fd4064afe530, user_id=6b22696ebeea4ff4b2eb37529ad3e429, role_id=6d053d567af44b6685295deb347f550e)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:01.020315 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39452c3-2afa-41d2-9a3f-1952a673c3a1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:01.040212 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3d11a-3784-4ee0-808b-4790bcd1d1ab None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:01.059635 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 94/177] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:00 2022] PUT /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530/users/6b22696ebeea4ff4b2eb37529ad3e429/roles/6d053d567af44b6685295deb347f550e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:28:01.078319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6457b-6a51-4e2e-a4ff-a5c17e8a7c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:01.079172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6457b-6a51-4e2e-a4ff-a5c17e8a7c0d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:01.079533 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6457b-6a51-4e2e-a4ff-a5c17e8a7c0d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:01.079892 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6457b-6a51-4e2e-a4ff-a5c17e8a7c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:01.180897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6457b-6a51-4e2e-a4ff-a5c17e8a7c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:01.182474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6457b-6a51-4e2e-a4ff-a5c17e8a7c0d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:01.294671 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 84/178] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:28:01.300630 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39452c3-2afa-41d2-9a3f-1952a673c3a1 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:01.424595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39452c3-2afa-41d2-9a3f-1952a673c3a1 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:01.466736 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac61ce-2d6e-44d6-ae6b-e2446b0b72cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:01.467964 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac61ce-2d6e-44d6-ae6b-e2446b0b72cd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:01.468471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac61ce-2d6e-44d6-ae6b-e2446b0b72cd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:01.468948 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac61ce-2d6e-44d6-ae6b-e2446b0b72cd None admin] PATH_INFO: `/v3/projects/5874c0dfc2ac4581bb3cf5dc4a692c54` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:01.470395 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac61ce-2d6e-44d6-ae6b-e2446b0b72cd None admin] RBAC: Authorizing `identity:get_project(project_id=5874c0dfc2ac4581bb3cf5dc4a692c54)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:01.472043 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac61ce-2d6e-44d6-ae6b-e2446b0b72cd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:01.474524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 95/179] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:01 2022] GET /identity/v3/projects/5874c0dfc2ac4581bb3cf5dc4a692c54 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:01.480457 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6457b-6a51-4e2e-a4ff-a5c17e8a7c0d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:01.606283 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6457b-6a51-4e2e-a4ff-a5c17e8a7c0d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:01.640858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fde880-fea1-4226-993a-e05b2d3c7639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:01.641556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fde880-fea1-4226-993a-e05b2d3c7639 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:01.641845 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fde880-fea1-4226-993a-e05b2d3c7639 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:01.642128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fde880-fea1-4226-993a-e05b2d3c7639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:01.642985 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fde880-fea1-4226-993a-e05b2d3c7639 None admin] RBAC: Adding query filter params (role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=5874c0dfc2ac4581bb3cf5dc4a692c54, user.id=0fe86d9a6c224cacaa25f0d29d448f20) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:01.643305 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fde880-fea1-4226-993a-e05b2d3c7639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:01.644861 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fde880-fea1-4226-993a-e05b2d3c7639 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:01.651187 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 85/180] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:01 2022] GET /identity/v3/role_assignments?user.id=0fe86d9a6c224cacaa25f0d29d448f20&scope.project.id=5874c0dfc2ac4581bb3cf5dc4a692c54&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:02.748940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027c3f-be00-4351-ad3b-3bc642633d23 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:02.750028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027c3f-be00-4351-ad3b-3bc642633d23 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:02.750535 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db027c3f-be00-4351-ad3b-3bc642633d23 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:02.752145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 96/181] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:02.758293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7dd40-d162-48ef-8371-334dbfb511f7 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:02.759150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7dd40-d162-48ef-8371-334dbfb511f7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:02.759653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7dd40-d162-48ef-8371-334dbfb511f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:02.776608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-a9c7dd40-d162-48ef-8371-334dbfb511f7 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:02.778138 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-a9c7dd40-d162-48ef-8371-334dbfb511f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:02.845172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 86/182] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:02.869871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db027c3f-be00-4351-ad3b-3bc642633d23 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:02.979045 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db027c3f-be00-4351-ad3b-3bc642633d23 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.013266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7e1fc-3819-4b6c-b9ab-7acd431b6e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.014408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7e1fc-3819-4b6c-b9ab-7acd431b6e57 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.014912 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7e1fc-3819-4b6c-b9ab-7acd431b6e57 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.015392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7e1fc-3819-4b6c-b9ab-7acd431b6e57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.025917 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7e1fc-3819-4b6c-b9ab-7acd431b6e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-33d7e1fc-3819-4b6c-b9ab-7acd431b6e57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:03.033965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:03.035016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:03.036432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:03.039486 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 97/183] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:03.045819 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7dd40-d162-48ef-8371-334dbfb511f7 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.073463 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a324693-4c92-4958-9014-ca65e4913c3a None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.073819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a324693-4c92-4958-9014-ca65e4913c3a None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.074033 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a324693-4c92-4958-9014-ca65e4913c3a None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.075454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 98/184] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:03.080786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcf58a-21d9-42c9-a623-b1ecac95ddf3 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.081107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcf58a-21d9-42c9-a623-b1ecac95ddf3 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.081380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcf58a-21d9-42c9-a623-b1ecac95ddf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.109192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-d1bcf58a-21d9-42c9-a623-b1ecac95ddf3 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:03.110594 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-d1bcf58a-21d9-42c9-a623-b1ecac95ddf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:03.154210 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7dd40-d162-48ef-8371-334dbfb511f7 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.180406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 99/185] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:03.193924 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.194973 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.195440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.195896 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.196850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:03.197380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:03.199025 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:03.204434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcf58a-21d9-42c9-a623-b1ecac95ddf3 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.211388 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 87/186] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:03.218457 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.327182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcf58a-21d9-42c9-a623-b1ecac95ddf3 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.333474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69514d08-39e4-422c-bc93-4269602233ee None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.362293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfd621-18fb-4628-a938-25484d611488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.363417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfd621-18fb-4628-a938-25484d611488 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.363746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfd621-18fb-4628-a938-25484d611488 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.364021 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfd621-18fb-4628-a938-25484d611488 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.366633 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099fda7-ad07-4bf2-b68b-2fb3c6ad7c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.367501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099fda7-ad07-4bf2-b68b-2fb3c6ad7c89 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.367790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099fda7-ad07-4bf2-b68b-2fb3c6ad7c89 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.368071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099fda7-ad07-4bf2-b68b-2fb3c6ad7c89 None admin] PATH_INFO: `/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.374930 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfd621-18fb-4628-a938-25484d611488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:03.381475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099fda7-ad07-4bf2-b68b-2fb3c6ad7c89 None admin] RBAC: Authorizing `identity:get_user(user_id=3688fa7bc5ce4c528e3def96e48cc0ad)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:03.382958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099fda7-ad07-4bf2-b68b-2fb3c6ad7c89 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:03.384815 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 88/187] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:03 2022] GET /identity/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:03.389637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcae61-b79e-4616-88d1-d0a0dad804c4 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.389637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcae61-b79e-4616-88d1-d0a0dad804c4 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.389637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcae61-b79e-4616-88d1-d0a0dad804c4 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.389637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 89/188] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:03.394144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcae61-b79e-4616-88d1-d0a0dad804c4 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-71cfd621-18fb-4628-a938-25484d611488 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:03.394282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:03.395745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 100/189] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:03.396615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc57a-35b4-4e13-936b-d8408ceaff20 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.398839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc57a-35b4-4e13-936b-d8408ceaff20 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.398839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc57a-35b4-4e13-936b-d8408ceaff20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.414613 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-63afc57a-35b4-4e13-936b-d8408ceaff20 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:03.415798 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-63afc57a-35b4-4e13-936b-d8408ceaff20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:03.489202 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 101/190] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:03.490084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afc57a-35b4-4e13-936b-d8408ceaff20 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.509946 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcae61-b79e-4616-88d1-d0a0dad804c4 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.548001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616b1c3-104d-45b3-9f9d-fe429a1ab633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.548667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616b1c3-104d-45b3-9f9d-fe429a1ab633 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.548957 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616b1c3-104d-45b3-9f9d-fe429a1ab633 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.549262 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616b1c3-104d-45b3-9f9d-fe429a1ab633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.598109 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afc57a-35b4-4e13-936b-d8408ceaff20 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.619040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616b1c3-104d-45b3-9f9d-fe429a1ab633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:03.620519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616b1c3-104d-45b3-9f9d-fe429a1ab633 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:03.632218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.632897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.633218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.633501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.634269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:03.634581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:03.635962 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:03.644170 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 102/191] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:03.645806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.725146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 90/192] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:03.725861 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616b1c3-104d-45b3-9f9d-fe429a1ab633 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.748817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747f618-78f6-442f-816f-3e251a7b8942 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.781509 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944523a-3df3-47a9-9cb5-d2b1c3ba2a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.782305 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944523a-3df3-47a9-9cb5-d2b1c3ba2a27 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.782590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944523a-3df3-47a9-9cb5-d2b1c3ba2a27 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.782871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944523a-3df3-47a9-9cb5-d2b1c3ba2a27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.784065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944523a-3df3-47a9-9cb5-d2b1c3ba2a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:03.785662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944523a-3df3-47a9-9cb5-d2b1c3ba2a27 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:03.787325 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 103/193] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:03.788493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c4332-32fe-47f1-96fe-32372bac15ea None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.788783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c4332-32fe-47f1-96fe-32372bac15ea None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.789088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c4332-32fe-47f1-96fe-32372bac15ea None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.790547 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 104/194] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:03.791181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c4332-32fe-47f1-96fe-32372bac15ea None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.820608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616b1c3-104d-45b3-9f9d-fe429a1ab633 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.853430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b01ab-5258-46cf-9652-97d316af1e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.854223 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b01ab-5258-46cf-9652-97d316af1e41 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.854505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b01ab-5258-46cf-9652-97d316af1e41 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.854777 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b01ab-5258-46cf-9652-97d316af1e41 None admin] PATH_INFO: `/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.855979 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b01ab-5258-46cf-9652-97d316af1e41 None admin] RBAC: Authorizing `identity:get_user(user_id=3688fa7bc5ce4c528e3def96e48cc0ad)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:03.857734 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b01ab-5258-46cf-9652-97d316af1e41 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:03.859360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 91/195] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:03 2022] GET /identity/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad => generated 283 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:03.860101 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b01ab-5258-46cf-9652-97d316af1e41 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:03.892855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c4332-32fe-47f1-96fe-32372bac15ea None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.924545 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd0831-e8b6-4163-808c-d339e70894da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.925429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0831-e8b6-4163-808c-d339e70894da None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.925712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0831-e8b6-4163-808c-d339e70894da None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.926017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0831-e8b6-4163-808c-d339e70894da None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.927191 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd0831-e8b6-4163-808c-d339e70894da None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:03.928636 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd0831-e8b6-4163-808c-d339e70894da None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:03.930366 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 105/196] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:03 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:03.931574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380cf43-2176-4bf8-9935-777f58e43291 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.931862 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380cf43-2176-4bf8-9935-777f58e43291 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.932136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380cf43-2176-4bf8-9935-777f58e43291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.948688 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-3380cf43-2176-4bf8-9935-777f58e43291 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:03.949913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-3380cf43-2176-4bf8-9935-777f58e43291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:03.954902 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b01ab-5258-46cf-9652-97d316af1e41 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:03.987092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f2d3-b18d-4162-9e95-0a6ae811d74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:03.987758 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f2d3-b18d-4162-9e95-0a6ae811d74f None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:03.988053 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f2d3-b18d-4162-9e95-0a6ae811d74f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:03.988834 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955f2d3-b18d-4162-9e95-0a6ae811d74f None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:03.989597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955f2d3-b18d-4162-9e95-0a6ae811d74f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:03.991043 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955f2d3-b18d-4162-9e95-0a6ae811d74f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:04.009863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 92/197] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:03 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:04.010672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f2d3-b18d-4162-9e95-0a6ae811d74f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:04.025838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 106/198] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:04.026746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380cf43-2176-4bf8-9935-777f58e43291 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:04.120893 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955f2d3-b18d-4162-9e95-0a6ae811d74f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:04.141379 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380cf43-2176-4bf8-9935-777f58e43291 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:04.155832 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b101-ef83-4af2-bfd2-d1baafc05742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:04.156561 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b101-ef83-4af2-bfd2-d1baafc05742 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:04.156853 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b101-ef83-4af2-bfd2-d1baafc05742 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:04.157173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18b101-ef83-4af2-bfd2-d1baafc05742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:04.176982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:04.177697 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:04.177982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:04.178257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:04.179120 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=d6cb7aefad52496890bce00f2bb83781, user.id=3688fa7bc5ce4c528e3def96e48cc0ad) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:04.179437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:04.180937 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:04.186975 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 107/199] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:04 2022] GET /identity/v3/role_assignments?user.id=3688fa7bc5ce4c528e3def96e48cc0ad&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=58c34600086c469b87bfa39b74dd2607 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:04.194129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:04.231203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18b101-ef83-4af2-bfd2-d1baafc05742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:04.232959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18b101-ef83-4af2-bfd2-d1baafc05742 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:04.330847 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd507b25-bd06-4ef0-876f-d8571072eafd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:04.342565 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 93/200] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:04.349149 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b101-ef83-4af2-bfd2-d1baafc05742 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:04.370236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827ac51-7807-48b8-a823-326b139340d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:04.371399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827ac51-7807-48b8-a823-326b139340d5 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:04.371834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827ac51-7807-48b8-a823-326b139340d5 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:04.372277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827ac51-7807-48b8-a823-326b139340d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:04.383939 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827ac51-7807-48b8-a823-326b139340d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-f827ac51-7807-48b8-a823-326b139340d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:04.391967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:04.393206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:04.394221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:04.396338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 108/201] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:04.407647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827ac51-7807-48b8-a823-326b139340d5 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:04.456882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18b101-ef83-4af2-bfd2-d1baafc05742 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:04.496601 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313880af-76c7-4759-bc3e-5dad52ca0f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:04.497614 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313880af-76c7-4759-bc3e-5dad52ca0f41 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:04.497793 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313880af-76c7-4759-bc3e-5dad52ca0f41 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:04.498078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313880af-76c7-4759-bc3e-5dad52ca0f41 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:04.499325 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313880af-76c7-4759-bc3e-5dad52ca0f41 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:04.500823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313880af-76c7-4759-bc3e-5dad52ca0f41 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:04.502914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 94/202] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:04 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:04.508977 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313880af-76c7-4759-bc3e-5dad52ca0f41 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:04.515529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827ac51-7807-48b8-a823-326b139340d5 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:04.554575 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:04.555581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:04.556070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:04.556503 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:04.557443 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:04.558001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:04.559702 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:04.568580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 109/203] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:04.573978 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:04.633346 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313880af-76c7-4759-bc3e-5dad52ca0f41 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:04.677584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:04.678276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:04.678568 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:04.678853 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:04.680158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] RBAC: Adding query filter params (role.id=02f09975329342eead41c01b150325c0, scope.project.id=d6cb7aefad52496890bce00f2bb83781, user.id=3688fa7bc5ce4c528e3def96e48cc0ad) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:04.680483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:04.682269 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:04.689486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 95/204] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:04 2022] GET /identity/v3/role_assignments?user.id=3688fa7bc5ce4c528e3def96e48cc0ad&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=02f09975329342eead41c01b150325c0 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:04.723758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9405ad-ad9f-49ed-817f-170988cca5cb None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:04.766097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5914a-0937-41f2-ad16-ee1cf2275150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:04.767314 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5914a-0937-41f2-ad16-ee1cf2275150 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:04.767782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5914a-0937-41f2-ad16-ee1cf2275150 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:04.768219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5914a-0937-41f2-ad16-ee1cf2275150 None admin] PATH_INFO: `/v3/users/6b22696ebeea4ff4b2eb37529ad3e429` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:04.770497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5914a-0937-41f2-ad16-ee1cf2275150 None admin] RBAC: Authorizing `identity:get_user(user_id=6b22696ebeea4ff4b2eb37529ad3e429)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:04.772266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5914a-0937-41f2-ad16-ee1cf2275150 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:04.774689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 110/205] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:04 2022] GET /identity/v3/users/6b22696ebeea4ff4b2eb37529ad3e429 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:04.782562 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:04.934866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71e2e-65c4-470c-b3ea-7d47b433029e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:04.970194 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8ae7-bccb-4af9-aa47-bf809c8a46fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:04.970861 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8ae7-bccb-4af9-aa47-bf809c8a46fe None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:04.971151 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8ae7-bccb-4af9-aa47-bf809c8a46fe None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:04.971437 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8ae7-bccb-4af9-aa47-bf809c8a46fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:05.043986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af8ae7-bccb-4af9-aa47-bf809c8a46fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:05.045520 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af8ae7-bccb-4af9-aa47-bf809c8a46fe None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:05.151750 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 96/206] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:05.159104 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5914a-0937-41f2-ad16-ee1cf2275150 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:05.290566 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5914a-0937-41f2-ad16-ee1cf2275150 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:05.330444 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccc4ad-fff5-4bb7-bfdf-6ce52fa4223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:05.331669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccc4ad-fff5-4bb7-bfdf-6ce52fa4223d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:05.332280 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccc4ad-fff5-4bb7-bfdf-6ce52fa4223d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:05.332813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccc4ad-fff5-4bb7-bfdf-6ce52fa4223d None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:05.337758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccc4ad-fff5-4bb7-bfdf-6ce52fa4223d None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:05.339448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccc4ad-fff5-4bb7-bfdf-6ce52fa4223d None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:05.341426 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 111/207] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:05 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:05.347267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8ae7-bccb-4af9-aa47-bf809c8a46fe None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:05.464475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8ae7-bccb-4af9-aa47-bf809c8a46fe None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:05.504602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34adc-c077-40a6-a23c-98f8d17a0d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:05.505309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34adc-c077-40a6-a23c-98f8d17a0d8d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:05.505651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34adc-c077-40a6-a23c-98f8d17a0d8d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:05.505940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34adc-c077-40a6-a23c-98f8d17a0d8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:05.506860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34adc-c077-40a6-a23c-98f8d17a0d8d None admin] RBAC: Adding query filter params (role.id=6d053d567af44b6685295deb347f550e, scope.project.id=3d209aef7bbd4477a103fd4064afe530, user.id=6b22696ebeea4ff4b2eb37529ad3e429) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:05.507185 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34adc-c077-40a6-a23c-98f8d17a0d8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:05.508712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34adc-c077-40a6-a23c-98f8d17a0d8d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:05.515569 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 97/208] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:05 2022] GET /identity/v3/role_assignments?user.id=6b22696ebeea4ff4b2eb37529ad3e429&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=6d053d567af44b6685295deb347f550e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:06.628169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cf015-6ecb-4f74-adbd-769af3a8c2e2 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:06.629819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cf015-6ecb-4f74-adbd-769af3a8c2e2 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:06.630364 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cf015-6ecb-4f74-adbd-769af3a8c2e2 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:06.632301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 112/209] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:06.638448 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef1021-6fa8-4842-b1b3-b26efa004754 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:06.639329 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef1021-6fa8-4842-b1b3-b26efa004754 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:06.639822 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef1021-6fa8-4842-b1b3-b26efa004754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:06.657819 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-26ef1021-6fa8-4842-b1b3-b26efa004754 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:06.659368 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-26ef1021-6fa8-4842-b1b3-b26efa004754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:06.733749 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 98/210] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:06.756493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cf015-6ecb-4f74-adbd-769af3a8c2e2 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:06.793259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321eaa-881d-4ebd-affa-673704d55790 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:06.793594 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321eaa-881d-4ebd-affa-673704d55790 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:06.793823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321eaa-881d-4ebd-affa-673704d55790 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:06.795486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 99/211] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:06.802335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a78d4-06e8-4143-91ce-bd16e6ac560d None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:06.802742 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a78d4-06e8-4143-91ce-bd16e6ac560d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:06.803131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a78d4-06e8-4143-91ce-bd16e6ac560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:06.820224 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-ea9a78d4-06e8-4143-91ce-bd16e6ac560d None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:06.821592 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-ea9a78d4-06e8-4143-91ce-bd16e6ac560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:06.865027 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cf015-6ecb-4f74-adbd-769af3a8c2e2 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:06.895218 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 100/212] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:06.907102 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1e8cf-dcbb-477d-bac2-cc5d8d489364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:06.907913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1e8cf-dcbb-477d-bac2-cc5d8d489364 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:06.908220 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1e8cf-dcbb-477d-bac2-cc5d8d489364 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:06.908503 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1e8cf-dcbb-477d-bac2-cc5d8d489364 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:06.919501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1e8cf-dcbb-477d-bac2-cc5d8d489364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-57f1e8cf-dcbb-477d-bac2-cc5d8d489364 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:06.927334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:06.929674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:06.931708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 113/213] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:28:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:06.934463 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a78d4-06e8-4143-91ce-bd16e6ac560d None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:06.937285 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1e8cf-dcbb-477d-bac2-cc5d8d489364 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.051708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1e8cf-dcbb-477d-bac2-cc5d8d489364 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.057793 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a78d4-06e8-4143-91ce-bd16e6ac560d None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.091829 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde2587-1da6-436f-a129-080f2028fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.092640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde2587-1da6-436f-a129-080f2028fb14 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.092923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde2587-1da6-436f-a129-080f2028fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.095935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde2587-1da6-436f-a129-080f2028fb14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.102203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95a498-9bd5-4a31-9c7a-1ee2d961fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.102879 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95a498-9bd5-4a31-9c7a-1ee2d961fb39 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.103202 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95a498-9bd5-4a31-9c7a-1ee2d961fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.103422 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95a498-9bd5-4a31-9c7a-1ee2d961fb39 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.104181 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95a498-9bd5-4a31-9c7a-1ee2d961fb39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:07.104484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95a498-9bd5-4a31-9c7a-1ee2d961fb39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.105909 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95a498-9bd5-4a31-9c7a-1ee2d961fb39 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.107897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde2587-1da6-436f-a129-080f2028fb14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:07.114984 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 101/214] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-4cde2587-1da6-436f-a129-080f2028fb14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:07.116127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:07.117128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 114/215] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:07.122873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde2587-1da6-436f-a129-080f2028fb14 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.123674 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec0806-3326-4b81-b35c-866b184c89d9 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.123674 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec0806-3326-4b81-b35c-866b184c89d9 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.123674 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec0806-3326-4b81-b35c-866b184c89d9 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.124649 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 102/216] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:07.125320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec0806-3326-4b81-b35c-866b184c89d9 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.231598 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec0806-3326-4b81-b35c-866b184c89d9 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.243136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde2587-1da6-436f-a129-080f2028fb14 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.272761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711d4a1-ce4c-4238-bbd6-c012032b7747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.273921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d4a1-ce4c-4238-bbd6-c012032b7747 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.274358 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d4a1-ce4c-4238-bbd6-c012032b7747 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.274788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d4a1-ce4c-4238-bbd6-c012032b7747 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.275976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711d4a1-ce4c-4238-bbd6-c012032b7747 None admin] RBAC: Adding query filter params (name=member) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:07.276467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711d4a1-ce4c-4238-bbd6-c012032b7747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.278701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711d4a1-ce4c-4238-bbd6-c012032b7747 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.290972 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 103/217] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:07.291352 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ff5a7-8061-4be4-9414-a53eb942a694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.292473 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ff5a7-8061-4be4-9414-a53eb942a694 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.292777 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13871-01e8-4456-a346-cb34b0e03a76 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.293111 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ff5a7-8061-4be4-9414-a53eb942a694 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.293231 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13871-01e8-4456-a346-cb34b0e03a76 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.293408 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b13871-01e8-4456-a346-cb34b0e03a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.293769 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ff5a7-8061-4be4-9414-a53eb942a694 None admin] PATH_INFO: `/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.296548 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ff5a7-8061-4be4-9414-a53eb942a694 None admin] RBAC: Authorizing `identity:get_user(user_id=3688fa7bc5ce4c528e3def96e48cc0ad)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.298062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ff5a7-8061-4be4-9414-a53eb942a694 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.299851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 115/218] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:07 2022] GET /identity/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:07.304793 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ff5a7-8061-4be4-9414-a53eb942a694 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.317933 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-79b13871-01e8-4456-a346-cb34b0e03a76 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:07.319125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-79b13871-01e8-4456-a346-cb34b0e03a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:07.383233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 104/219] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:07.385220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b13871-01e8-4456-a346-cb34b0e03a76 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.400638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ff5a7-8061-4be4-9414-a53eb942a694 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.433756 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fd685-c51e-4973-85c1-e175ffcf5399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.434631 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fd685-c51e-4973-85c1-e175ffcf5399 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.434917 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fd685-c51e-4973-85c1-e175ffcf5399 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.435196 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fd685-c51e-4973-85c1-e175ffcf5399 None admin] PATH_INFO: `/v3/users/3943f8b44f12485c88f96089b3c438ef` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.448275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fd685-c51e-4973-85c1-e175ffcf5399 None admin] RBAC: Authorizing `identity:get_user(user_id=3943f8b44f12485c88f96089b3c438ef)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.449780 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fd685-c51e-4973-85c1-e175ffcf5399 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.451459 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 116/220] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:07 2022] GET /identity/v3/users/3943f8b44f12485c88f96089b3c438ef => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:07.452722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70cecd-d80d-4d66-8a3d-133b53fd2de0 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.453018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70cecd-d80d-4d66-8a3d-133b53fd2de0 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.453318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70cecd-d80d-4d66-8a3d-133b53fd2de0 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.454717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 117/221] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:07.455308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70cecd-d80d-4d66-8a3d-133b53fd2de0 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.486570 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b13871-01e8-4456-a346-cb34b0e03a76 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.519730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd375ac-b82c-47c4-94c7-f2236faf4add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.520610 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd375ac-b82c-47c4-94c7-f2236faf4add None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.520896 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd375ac-b82c-47c4-94c7-f2236faf4add None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.521213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd375ac-b82c-47c4-94c7-f2236faf4add None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.522400 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd375ac-b82c-47c4-94c7-f2236faf4add None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.523854 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd375ac-b82c-47c4-94c7-f2236faf4add None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.525614 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 105/222] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:07 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:07.526379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd375ac-b82c-47c4-94c7-f2236faf4add None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.554601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70cecd-d80d-4d66-8a3d-133b53fd2de0 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.586839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec802c-fb1e-413d-819e-b743ce17de11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.587636 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec802c-fb1e-413d-819e-b743ce17de11 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.587914 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec802c-fb1e-413d-819e-b743ce17de11 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.588183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec802c-fb1e-413d-819e-b743ce17de11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.589400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec802c-fb1e-413d-819e-b743ce17de11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.591340 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec802c-fb1e-413d-819e-b743ce17de11 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.592968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 118/223] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:07.594213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087d250-9faf-4abc-8fb6-4ba7e5c4902e None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.594497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087d250-9faf-4abc-8fb6-4ba7e5c4902e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.594802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087d250-9faf-4abc-8fb6-4ba7e5c4902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.611246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-2087d250-9faf-4abc-8fb6-4ba7e5c4902e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:07.612404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-2087d250-9faf-4abc-8fb6-4ba7e5c4902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:07.629651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd375ac-b82c-47c4-94c7-f2236faf4add None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.661862 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c658f16-b753-46bc-814e-8f1652b93de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.662522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c658f16-b753-46bc-814e-8f1652b93de7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.662807 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c658f16-b753-46bc-814e-8f1652b93de7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.663091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c658f16-b753-46bc-814e-8f1652b93de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.674852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 119/224] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:07.675752 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087d250-9faf-4abc-8fb6-4ba7e5c4902e None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.735105 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c658f16-b753-46bc-814e-8f1652b93de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.736642 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c658f16-b753-46bc-814e-8f1652b93de7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.769282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087d250-9faf-4abc-8fb6-4ba7e5c4902e None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.801704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246fd82-247b-4c80-beb2-fd514019894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.802589 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246fd82-247b-4c80-beb2-fd514019894e None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.802869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246fd82-247b-4c80-beb2-fd514019894e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.803139 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246fd82-247b-4c80-beb2-fd514019894e None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781/users/3688fa7bc5ce4c528e3def96e48cc0ad/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.805147 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246fd82-247b-4c80-beb2-fd514019894e None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cb7aefad52496890bce00f2bb83781, user_id=3688fa7bc5ce4c528e3def96e48cc0ad, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.806608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246fd82-247b-4c80-beb2-fd514019894e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.818416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 120/225] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:07 2022] PUT /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781/users/3688fa7bc5ce4c528e3def96e48cc0ad/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:07.819193 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246fd82-247b-4c80-beb2-fd514019894e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.829864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 106/226] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:07.830757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c658f16-b753-46bc-814e-8f1652b93de7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:07.937585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c658f16-b753-46bc-814e-8f1652b93de7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.943801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246fd82-247b-4c80-beb2-fd514019894e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:07.973478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a32b55-d22d-4133-a5a6-0fd3ec2cb4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.974308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32b55-d22d-4133-a5a6-0fd3ec2cb4a6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.974586 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32b55-d22d-4133-a5a6-0fd3ec2cb4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.974853 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32b55-d22d-4133-a5a6-0fd3ec2cb4a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.981170 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c62a3-c343-4de1-9ecf-ea867ca2112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:07.981859 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c62a3-c343-4de1-9ecf-ea867ca2112c None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:07.982148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c62a3-c343-4de1-9ecf-ea867ca2112c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:07.982692 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c62a3-c343-4de1-9ecf-ea867ca2112c None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:07.983433 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c62a3-c343-4de1-9ecf-ea867ca2112c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:07.984931 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c62a3-c343-4de1-9ecf-ea867ca2112c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:07.986378 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a32b55-d22d-4133-a5a6-0fd3ec2cb4a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-07a32b55-d22d-4133-a5a6-0fd3ec2cb4a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:07.994800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:07.995889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:07.996918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 107/227] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:07.999146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a32b55-d22d-4133-a5a6-0fd3ec2cb4a6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:08.014430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 121/228] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:07 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:08.021067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c62a3-c343-4de1-9ecf-ea867ca2112c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:08.120361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a32b55-d22d-4133-a5a6-0fd3ec2cb4a6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:08.135015 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c62a3-c343-4de1-9ecf-ea867ca2112c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:08.159375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93804ed8-7b50-49a9-bdcc-04c127fed2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:08.160302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804ed8-7b50-49a9-bdcc-04c127fed2cb None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:08.160651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804ed8-7b50-49a9-bdcc-04c127fed2cb None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:08.160940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804ed8-7b50-49a9-bdcc-04c127fed2cb None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:08.162232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93804ed8-7b50-49a9-bdcc-04c127fed2cb None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:08.163736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93804ed8-7b50-49a9-bdcc-04c127fed2cb None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:08.165565 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 108/229] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:07 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:08.171545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93804ed8-7b50-49a9-bdcc-04c127fed2cb None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:08.173337 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451f861-167b-4451-8100-8e7733813672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:08.174026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451f861-167b-4451-8100-8e7733813672 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:08.174312 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451f861-167b-4451-8100-8e7733813672 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:08.174592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451f861-167b-4451-8100-8e7733813672 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:08.175375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451f861-167b-4451-8100-8e7733813672 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:08.175690 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451f861-167b-4451-8100-8e7733813672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:08.177248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451f861-167b-4451-8100-8e7733813672 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:08.186869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 122/230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:08.193254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451f861-167b-4451-8100-8e7733813672 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:08.286068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93804ed8-7b50-49a9-bdcc-04c127fed2cb None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:08.317582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451f861-167b-4451-8100-8e7733813672 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:08.335094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:08.335864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:08.336153 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:08.336442 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:08.337374 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] RBAC: Adding query filter params (role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=d6cb7aefad52496890bce00f2bb83781, user.id=3943f8b44f12485c88f96089b3c438ef) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:08.337667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:08.339183 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:08.345087 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 109/231] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:08 2022] GET /identity/v3/role_assignments?user.id=3943f8b44f12485c88f96089b3c438ef&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:08.365080 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e483f-43d1-4f6e-9650-d2eaed86d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:08.365080 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e483f-43d1-4f6e-9650-d2eaed86d967 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:08.365080 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e483f-43d1-4f6e-9650-d2eaed86d967 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:08.365080 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e483f-43d1-4f6e-9650-d2eaed86d967 None admin] PATH_INFO: `/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:08.366354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e483f-43d1-4f6e-9650-d2eaed86d967 None admin] RBAC: Authorizing `identity:get_user(user_id=3688fa7bc5ce4c528e3def96e48cc0ad)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:08.367933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e483f-43d1-4f6e-9650-d2eaed86d967 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:08.370178 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 123/232] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:08 2022] GET /identity/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:08.375824 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:08.488134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20d6bf-27aa-4eb5-9fe0-2f9998d4f7c5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:08.525288 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60079701-b1ff-40e0-b258-7c647c7dd688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:08.526635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60079701-b1ff-40e0-b258-7c647c7dd688 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:08.527145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60079701-b1ff-40e0-b258-7c647c7dd688 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:08.527694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60079701-b1ff-40e0-b258-7c647c7dd688 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:08.529281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60079701-b1ff-40e0-b258-7c647c7dd688 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:08.531089 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60079701-b1ff-40e0-b258-7c647c7dd688 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:08.533475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 110/233] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:08 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:08.541289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e483f-43d1-4f6e-9650-d2eaed86d967 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:08.663664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e483f-43d1-4f6e-9650-d2eaed86d967 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:08.704165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ff60e-a7ea-4294-bed1-2ee7cd4c3e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:08.705084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ff60e-a7ea-4294-bed1-2ee7cd4c3e19 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:08.705377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ff60e-a7ea-4294-bed1-2ee7cd4c3e19 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:08.705657 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ff60e-a7ea-4294-bed1-2ee7cd4c3e19 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781/users/3688fa7bc5ce4c528e3def96e48cc0ad/roles/02f09975329342eead41c01b150325c0` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:08.708241 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ff60e-a7ea-4294-bed1-2ee7cd4c3e19 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cb7aefad52496890bce00f2bb83781, user_id=3688fa7bc5ce4c528e3def96e48cc0ad, role_id=02f09975329342eead41c01b150325c0)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:08.709801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ff60e-a7ea-4294-bed1-2ee7cd4c3e19 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:08.736604 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 124/234] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:08 2022] PUT /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781/users/3688fa7bc5ce4c528e3def96e48cc0ad/roles/02f09975329342eead41c01b150325c0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:10.441151 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a85ca-2000-44b6-bcc2-281b342b775f None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:10.442188 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a85ca-2000-44b6-bcc2-281b342b775f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:10.442694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a85ca-2000-44b6-bcc2-281b342b775f None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:10.444306 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 111/235] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:10.449867 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f74ec-8df6-4476-8530-cf327f2de848 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:10.450781 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f74ec-8df6-4476-8530-cf327f2de848 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:10.451282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f74ec-8df6-4476-8530-cf327f2de848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:10.478083 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-e95f74ec-8df6-4476-8530-cf327f2de848 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:10.479685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-e95f74ec-8df6-4476-8530-cf327f2de848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:10.555282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 125/236] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:10.590065 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a85ca-2000-44b6-bcc2-281b342b775f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:10.730778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a85ca-2000-44b6-bcc2-281b342b775f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:10.773064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c513e-c8e3-4698-853a-255ae381cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:10.773918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c513e-c8e3-4698-853a-255ae381cffe None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:10.774209 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c513e-c8e3-4698-853a-255ae381cffe None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:10.774492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c513e-c8e3-4698-853a-255ae381cffe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:10.799401 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c513e-c8e3-4698-853a-255ae381cffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-fe4c513e-c8e3-4698-853a-255ae381cffe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:10.807425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:10.808620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:10.810131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:10.813146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 112/237] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:28:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:10.817172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f74ec-8df6-4476-8530-cf327f2de848 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:10.940473 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f74ec-8df6-4476-8530-cf327f2de848 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:10.980902 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75457e6d-ab33-49e6-9ec9-8061b02b6e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:10.981556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75457e6d-ab33-49e6-9ec9-8061b02b6e15 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:10.981844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75457e6d-ab33-49e6-9ec9-8061b02b6e15 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:10.982119 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75457e6d-ab33-49e6-9ec9-8061b02b6e15 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:10.983182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75457e6d-ab33-49e6-9ec9-8061b02b6e15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:10.983496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75457e6d-ab33-49e6-9ec9-8061b02b6e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:10.984961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75457e6d-ab33-49e6-9ec9-8061b02b6e15 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:10.994759 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 126/238] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:11.000416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c513e-c8e3-4698-853a-255ae381cffe None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.077009 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63fb0-dfa3-4ad4-8d51-7587549dbede None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.077370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63fb0-dfa3-4ad4-8d51-7587549dbede None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.077595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63fb0-dfa3-4ad4-8d51-7587549dbede None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.079735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 127/239] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:11.090781 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907c0a-b5d8-46d4-a38d-88b900d8e67f None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.091079 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907c0a-b5d8-46d4-a38d-88b900d8e67f None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.091359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7907c0a-b5d8-46d4-a38d-88b900d8e67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.121674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-a7907c0a-b5d8-46d4-a38d-88b900d8e67f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:11.122868 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-a7907c0a-b5d8-46d4-a38d-88b900d8e67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:11.126375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c513e-c8e3-4698-853a-255ae381cffe None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.163246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127fe52-0ef6-4dbc-9bc0-e2d5616a143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.164136 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127fe52-0ef6-4dbc-9bc0-e2d5616a143e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.164427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127fe52-0ef6-4dbc-9bc0-e2d5616a143e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.164711 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127fe52-0ef6-4dbc-9bc0-e2d5616a143e None admin] PATH_INFO: `/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.165975 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127fe52-0ef6-4dbc-9bc0-e2d5616a143e None admin] RBAC: Authorizing `identity:get_user(user_id=3688fa7bc5ce4c528e3def96e48cc0ad)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:11.167643 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127fe52-0ef6-4dbc-9bc0-e2d5616a143e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:11.169809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 113/240] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:10 2022] GET /identity/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:11.171246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde6d7b-f442-48d6-b286-1f7421e9b8b7 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.171719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde6d7b-f442-48d6-b286-1f7421e9b8b7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.172155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde6d7b-f442-48d6-b286-1f7421e9b8b7 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.178181 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 114/241] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:11.179243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde6d7b-f442-48d6-b286-1f7421e9b8b7 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.213728 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 128/242] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:11.215385 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47b290-38fb-4211-967e-889434a72b67 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.215843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47b290-38fb-4211-967e-889434a72b67 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.216282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47b290-38fb-4211-967e-889434a72b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.240539 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-3b47b290-38fb-4211-967e-889434a72b67 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:11.242019 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-3b47b290-38fb-4211-967e-889434a72b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:11.293480 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde6d7b-f442-48d6-b286-1f7421e9b8b7 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.319058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 129/243] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:11.319058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d8474-a5a3-4328-864e-10613832e68e None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.319058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d8474-a5a3-4328-864e-10613832e68e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.319058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d8474-a5a3-4328-864e-10613832e68e None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.319058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 130/244] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:11.321843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d8474-a5a3-4328-864e-10613832e68e None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.330659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46888928-e8d8-4ab8-8cdf-5a4a736238dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.331324 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46888928-e8d8-4ab8-8cdf-5a4a736238dd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.331613 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46888928-e8d8-4ab8-8cdf-5a4a736238dd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.331893 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46888928-e8d8-4ab8-8cdf-5a4a736238dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.402030 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46888928-e8d8-4ab8-8cdf-5a4a736238dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:11.403508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46888928-e8d8-4ab8-8cdf-5a4a736238dd None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:11.430294 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d8474-a5a3-4328-864e-10613832e68e None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.466389 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65257755-bbb2-4829-b864-dd2497db8ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.467194 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257755-bbb2-4829-b864-dd2497db8ba0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.467477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257755-bbb2-4829-b864-dd2497db8ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.467750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257755-bbb2-4829-b864-dd2497db8ba0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.477915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65257755-bbb2-4829-b864-dd2497db8ba0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-65257755-bbb2-4829-b864-dd2497db8ba0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:11.485984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:11.486981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:11.487886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 131/245] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:11.495042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95f00-944d-4532-af09-6fc81144b8c5 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.495042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95f00-944d-4532-af09-6fc81144b8c5 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.495042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95f00-944d-4532-af09-6fc81144b8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.507831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 115/246] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:11.507831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46888928-e8d8-4ab8-8cdf-5a4a736238dd None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.508629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-79e95f00-944d-4532-af09-6fc81144b8c5 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:11.508815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-79e95f00-944d-4532-af09-6fc81144b8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:11.576802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 132/247] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:11.577748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e95f00-944d-4532-af09-6fc81144b8c5 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.604150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46888928-e8d8-4ab8-8cdf-5a4a736238dd None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.638296 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad28353-7524-4420-a1c4-15272e09193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.639093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad28353-7524-4420-a1c4-15272e09193f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.639380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad28353-7524-4420-a1c4-15272e09193f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.639657 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad28353-7524-4420-a1c4-15272e09193f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.649983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad28353-7524-4420-a1c4-15272e09193f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-1ad28353-7524-4420-a1c4-15272e09193f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:11.657701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:11.658693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:11.659725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:11.661388 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 116/248] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:11.662457 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad28353-7524-4420-a1c4-15272e09193f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.679160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e95f00-944d-4532-af09-6fc81144b8c5 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.712817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.713516 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.713802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.714078 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.714836 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:11.715147 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:11.716561 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:11.724404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 133/249] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:11.725943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.760355 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad28353-7524-4420-a1c4-15272e09193f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.791216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94847d98-0dae-4698-9bda-6cf6bdc4e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.792075 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94847d98-0dae-4698-9bda-6cf6bdc4e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.792359 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94847d98-0dae-4698-9bda-6cf6bdc4e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.792637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94847d98-0dae-4698-9bda-6cf6bdc4e0c0 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.793842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94847d98-0dae-4698-9bda-6cf6bdc4e0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:11.795284 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94847d98-0dae-4698-9bda-6cf6bdc4e0c0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:11.796975 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 117/250] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:11.802514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94847d98-0dae-4698-9bda-6cf6bdc4e0c0 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.821302 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4bfcb-5185-46c4-8b55-36495bbfd396 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.852558 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b513-b470-4a43-b9f0-958add0b400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.853384 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b513-b470-4a43-b9f0-958add0b400e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.853669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b513-b470-4a43-b9f0-958add0b400e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.854027 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b513-b470-4a43-b9f0-958add0b400e None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.855106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b513-b470-4a43-b9f0-958add0b400e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:11.856569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b513-b470-4a43-b9f0-958add0b400e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:11.858461 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 134/251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:11.859610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ed7b8-0f41-47b1-a50c-5a29c2935154 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.859900 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ed7b8-0f41-47b1-a50c-5a29c2935154 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.860177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ed7b8-0f41-47b1-a50c-5a29c2935154 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.861573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 135/252] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:11.865908 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ed7b8-0f41-47b1-a50c-5a29c2935154 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.892900 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94847d98-0dae-4698-9bda-6cf6bdc4e0c0 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.925008 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.925743 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.926520 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:11.926802 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:11.927560 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:11.927872 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:11.929292 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:11.937046 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 118/253] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:11.941646 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:11.965737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ed7b8-0f41-47b1-a50c-5a29c2935154 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:11.998718 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804619b5-e459-4e2d-bd05-e180eee9102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:11.999587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804619b5-e459-4e2d-bd05-e180eee9102e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:11.999874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804619b5-e459-4e2d-bd05-e180eee9102e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.000152 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804619b5-e459-4e2d-bd05-e180eee9102e None admin] PATH_INFO: `/v3/users/3943f8b44f12485c88f96089b3c438ef` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.001379 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804619b5-e459-4e2d-bd05-e180eee9102e None admin] RBAC: Authorizing `identity:get_user(user_id=3943f8b44f12485c88f96089b3c438ef)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.002840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804619b5-e459-4e2d-bd05-e180eee9102e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.004501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 136/254] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/users/3943f8b44f12485c88f96089b3c438ef => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:12.005273 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804619b5-e459-4e2d-bd05-e180eee9102e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.031899 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18057b79-bdec-4054-b37c-936419c2cc06 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.063794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60a65a-125a-4da8-8310-12cbc3cc95a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.064443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60a65a-125a-4da8-8310-12cbc3cc95a6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.064725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60a65a-125a-4da8-8310-12cbc3cc95a6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.065002 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60a65a-125a-4da8-8310-12cbc3cc95a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.065872 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60a65a-125a-4da8-8310-12cbc3cc95a6 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=d6cb7aefad52496890bce00f2bb83781, user.id=3688fa7bc5ce4c528e3def96e48cc0ad) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:12.066186 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60a65a-125a-4da8-8310-12cbc3cc95a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.067669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60a65a-125a-4da8-8310-12cbc3cc95a6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.073251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 119/255] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:11 2022] GET /identity/v3/role_assignments?user.id=3688fa7bc5ce4c528e3def96e48cc0ad&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=58c34600086c469b87bfa39b74dd2607 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:12.074492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baead932-2390-49aa-b7b8-01c64cf11e81 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.074787 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baead932-2390-49aa-b7b8-01c64cf11e81 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.075068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baead932-2390-49aa-b7b8-01c64cf11e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.096350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-baead932-2390-49aa-b7b8-01c64cf11e81 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:12.098187 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-baead932-2390-49aa-b7b8-01c64cf11e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:12.110108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804619b5-e459-4e2d-bd05-e180eee9102e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.145824 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d983df4-4540-4756-91cc-0d0dd3ff6e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.146518 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d983df4-4540-4756-91cc-0d0dd3ff6e9b None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.146801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d983df4-4540-4756-91cc-0d0dd3ff6e9b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.147077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d983df4-4540-4756-91cc-0d0dd3ff6e9b None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.147813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d983df4-4540-4756-91cc-0d0dd3ff6e9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.149319 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d983df4-4540-4756-91cc-0d0dd3ff6e9b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.181067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 137/256] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:12 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:12.181067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d983df4-4540-4756-91cc-0d0dd3ff6e9b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.181340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 120/257] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:12.182453 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baead932-2390-49aa-b7b8-01c64cf11e81 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.340611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baead932-2390-49aa-b7b8-01c64cf11e81 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.343544 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d983df4-4540-4756-91cc-0d0dd3ff6e9b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.391751 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dc18b-e5a6-46ba-b760-ff6c6000fc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.392624 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dc18b-e5a6-46ba-b760-ff6c6000fc0d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.392914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dc18b-e5a6-46ba-b760-ff6c6000fc0d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.393318 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26dc18b-e5a6-46ba-b760-ff6c6000fc0d None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.393528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ae48e-8341-40ae-ab76-8a924071a738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.394427 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ae48e-8341-40ae-ab76-8a924071a738 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.394698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26dc18b-e5a6-46ba-b760-ff6c6000fc0d None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.394813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ae48e-8341-40ae-ab76-8a924071a738 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.394977 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ae48e-8341-40ae-ab76-8a924071a738 None admin] PATH_INFO: `/v3/users/54f5af49266449b6a81b3ab46af8a1d3` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.396376 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26dc18b-e5a6-46ba-b760-ff6c6000fc0d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.398252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 121/258] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:12 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:12.399112 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dc18b-e5a6-46ba-b760-ff6c6000fc0d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.414027 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ae48e-8341-40ae-ab76-8a924071a738 None admin] RBAC: Authorizing `identity:get_user(user_id=54f5af49266449b6a81b3ab46af8a1d3)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.415539 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ae48e-8341-40ae-ab76-8a924071a738 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.417337 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 138/259] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:12 2022] GET /identity/v3/users/54f5af49266449b6a81b3ab46af8a1d3 => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:12.419159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ae48e-8341-40ae-ab76-8a924071a738 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.513182 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26dc18b-e5a6-46ba-b760-ff6c6000fc0d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.523998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ae48e-8341-40ae-ab76-8a924071a738 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.553163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51ceee-9373-4f7a-bee0-9cc09d1cdcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.553989 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51ceee-9373-4f7a-bee0-9cc09d1cdcf8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.554272 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51ceee-9373-4f7a-bee0-9cc09d1cdcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.554547 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51ceee-9373-4f7a-bee0-9cc09d1cdcf8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.561339 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd83591-1652-4534-a310-9284cc8a21f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.562233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd83591-1652-4534-a310-9284cc8a21f9 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.562514 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd83591-1652-4534-a310-9284cc8a21f9 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.562788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd83591-1652-4534-a310-9284cc8a21f9 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781/users/3943f8b44f12485c88f96089b3c438ef/roles/251b2e26a5834411bf902ba50b5ec81f` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.567029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd83591-1652-4534-a310-9284cc8a21f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cb7aefad52496890bce00f2bb83781, user_id=3943f8b44f12485c88f96089b3c438ef, role_id=251b2e26a5834411bf902ba50b5ec81f)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.568719 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd83591-1652-4534-a310-9284cc8a21f9 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.568978 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51ceee-9373-4f7a-bee0-9cc09d1cdcf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-ea51ceee-9373-4f7a-bee0-9cc09d1cdcf8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:12.577813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:12.578882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:12.579740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 122/260] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:12.580472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51ceee-9373-4f7a-bee0-9cc09d1cdcf8 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.591665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 139/261] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:12 2022] PUT /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781/users/3943f8b44f12485c88f96089b3c438ef/roles/251b2e26a5834411bf902ba50b5ec81f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:12.593880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd83591-1652-4534-a310-9284cc8a21f9 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.688631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51ceee-9373-4f7a-bee0-9cc09d1cdcf8 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.705593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd83591-1652-4534-a310-9284cc8a21f9 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.726115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b935c33-f02d-4a70-bce4-7360c7498fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.726971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b935c33-f02d-4a70-bce4-7360c7498fd3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.727362 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b935c33-f02d-4a70-bce4-7360c7498fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.727744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b935c33-f02d-4a70-bce4-7360c7498fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.745785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c98180-e968-4692-9504-34419cd79baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.746732 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c98180-e968-4692-9504-34419cd79baf None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.747166 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c98180-e968-4692-9504-34419cd79baf None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.747745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c98180-e968-4692-9504-34419cd79baf None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.748739 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c98180-e968-4692-9504-34419cd79baf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:12.749284 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c98180-e968-4692-9504-34419cd79baf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.751014 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c98180-e968-4692-9504-34419cd79baf None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.760695 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 140/262] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:12.766466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c98180-e968-4692-9504-34419cd79baf None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.800384 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b935c33-f02d-4a70-bce4-7360c7498fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.802680 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b935c33-f02d-4a70-bce4-7360c7498fd3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.873332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c98180-e968-4692-9504-34419cd79baf None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:12.907291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 123/263] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:12.909181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb41f35-f9fe-424e-a8a0-e9116069371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:12.910691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb41f35-f9fe-424e-a8a0-e9116069371e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:12.910979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb41f35-f9fe-424e-a8a0-e9116069371e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:12.911254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb41f35-f9fe-424e-a8a0-e9116069371e None admin] PATH_INFO: `/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:12.912567 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb41f35-f9fe-424e-a8a0-e9116069371e None admin] RBAC: Authorizing `identity:get_user(user_id=3688fa7bc5ce4c528e3def96e48cc0ad)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:12.913403 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b935c33-f02d-4a70-bce4-7360c7498fd3 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:12.914103 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb41f35-f9fe-424e-a8a0-e9116069371e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:12.916081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 141/264] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:12 2022] GET /identity/v3/users/3688fa7bc5ce4c528e3def96e48cc0ad => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:12.921374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb41f35-f9fe-424e-a8a0-e9116069371e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:13.032753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b935c33-f02d-4a70-bce4-7360c7498fd3 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:13.039032 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb41f35-f9fe-424e-a8a0-e9116069371e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:13.097068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38136ef-0496-4d4c-8e45-407ecb49f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:13.097068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38136ef-0496-4d4c-8e45-407ecb49f429 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:13.097068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38136ef-0496-4d4c-8e45-407ecb49f429 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:13.097068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38136ef-0496-4d4c-8e45-407ecb49f429 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:13.101282 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38136ef-0496-4d4c-8e45-407ecb49f429 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:13.101282 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38136ef-0496-4d4c-8e45-407ecb49f429 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:13.103848 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 124/265] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:12 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:13.112893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29802b8-82e7-415f-9455-a8d5450f7b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:13.113699 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29802b8-82e7-415f-9455-a8d5450f7b1f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:13.113959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38136ef-0496-4d4c-8e45-407ecb49f429 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:13.114093 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29802b8-82e7-415f-9455-a8d5450f7b1f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:13.114416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29802b8-82e7-415f-9455-a8d5450f7b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:13.232849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38136ef-0496-4d4c-8e45-407ecb49f429 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:13.264198 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29802b8-82e7-415f-9455-a8d5450f7b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:13.265725 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29802b8-82e7-415f-9455-a8d5450f7b1f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:13.296760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:13.297456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:13.297737 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:13.298011 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:13.298857 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] RBAC: Adding query filter params (role.id=6d053d567af44b6685295deb347f550e, scope.project.id=d6cb7aefad52496890bce00f2bb83781, user.id=54f5af49266449b6a81b3ab46af8a1d3) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:13.299168 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:13.301138 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:13.313856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 125/266] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:13 2022] GET /identity/v3/role_assignments?user.id=54f5af49266449b6a81b3ab46af8a1d3&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=6d053d567af44b6685295deb347f550e => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:13.423239 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 142/267] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:28:13.429546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:13.532711 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8e5aa-d3a4-4ba8-8a36-d18398de3e24 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:13.568763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2c8ba-a726-449f-9d30-096571046426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:13.569678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c8ba-a726-449f-9d30-096571046426 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:13.569958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c8ba-a726-449f-9d30-096571046426 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:13.570233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c8ba-a726-449f-9d30-096571046426 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:13.571490 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2c8ba-a726-449f-9d30-096571046426 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:13.572986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2c8ba-a726-449f-9d30-096571046426 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:13.574801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 126/268] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:13 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:13.580761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29802b8-82e7-415f-9455-a8d5450f7b1f None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:13.686785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29802b8-82e7-415f-9455-a8d5450f7b1f None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:13.721663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79929906-945b-439b-b6e8-2b291bf933a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:13.722322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79929906-945b-439b-b6e8-2b291bf933a8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:13.723229 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79929906-945b-439b-b6e8-2b291bf933a8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:13.723503 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79929906-945b-439b-b6e8-2b291bf933a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:13.724346 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79929906-945b-439b-b6e8-2b291bf933a8 None admin] RBAC: Adding query filter params (role.id=02f09975329342eead41c01b150325c0, scope.project.id=d6cb7aefad52496890bce00f2bb83781, user.id=3688fa7bc5ce4c528e3def96e48cc0ad) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:13.724658 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79929906-945b-439b-b6e8-2b291bf933a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:13.726134 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79929906-945b-439b-b6e8-2b291bf933a8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:13.732515 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 143/269] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:13 2022] GET /identity/v3/role_assignments?user.id=3688fa7bc5ce4c528e3def96e48cc0ad&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=02f09975329342eead41c01b150325c0 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:14.860380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cafb7-5cd0-494c-a7f2-ac94da9a9285 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:14.860910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cafb7-5cd0-494c-a7f2-ac94da9a9285 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:14.860910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cafb7-5cd0-494c-a7f2-ac94da9a9285 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:14.862831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 127/270] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:14.868766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45eb5-e91b-425f-95ab-d10cc7531e4e None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:14.869225 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45eb5-e91b-425f-95ab-d10cc7531e4e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:14.869426 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45eb5-e91b-425f-95ab-d10cc7531e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:14.886227 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-43c45eb5-e91b-425f-95ab-d10cc7531e4e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:14.887415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-43c45eb5-e91b-425f-95ab-d10cc7531e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:14.949690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620b7f-8e21-4690-894a-540955533062 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:14.950361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620b7f-8e21-4690-894a-540955533062 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:14.950873 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7620b7f-8e21-4690-894a-540955533062 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:14.953953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 128/271] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:14.958380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8cb7-b01a-4ddb-a795-97c3118aa09d None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:14.958680 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 144/272] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:14.959138 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8cb7-b01a-4ddb-a795-97c3118aa09d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:14.959496 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8cb7-b01a-4ddb-a795-97c3118aa09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:14.977079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-0fde8cb7-b01a-4ddb-a795-97c3118aa09d None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:14.978425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-0fde8cb7-b01a-4ddb-a795-97c3118aa09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:14.984085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45eb5-e91b-425f-95ab-d10cc7531e4e None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.053186 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 129/273] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:15.071668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8cb7-b01a-4ddb-a795-97c3118aa09d None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.083340 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45eb5-e91b-425f-95ab-d10cc7531e4e None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:15.122552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f457f-83a4-44b7-9145-57a6c9e817d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:15.122954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f457f-83a4-44b7-9145-57a6c9e817d6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.123358 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f457f-83a4-44b7-9145-57a6c9e817d6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.123785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f457f-83a4-44b7-9145-57a6c9e817d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.135705 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f457f-83a4-44b7-9145-57a6c9e817d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-cd3f457f-83a4-44b7-9145-57a6c9e817d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:15.143656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:15.144913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:15.145851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:15.146816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 145/274] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:15.151883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f457f-83a4-44b7-9145-57a6c9e817d6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.195007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8cb7-b01a-4ddb-a795-97c3118aa09d None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:15.230855 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e290c-59d0-4566-8343-4bd536a97fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:15.232000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e290c-59d0-4566-8343-4bd536a97fa1 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.232748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e290c-59d0-4566-8343-4bd536a97fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.233243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e290c-59d0-4566-8343-4bd536a97fa1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.234652 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e290c-59d0-4566-8343-4bd536a97fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:15.236330 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e290c-59d0-4566-8343-4bd536a97fa1 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:15.242850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 130/275] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:15.244396 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e290c-59d0-4566-8343-4bd536a97fa1 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.253723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f457f-83a4-44b7-9145-57a6c9e817d6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:15.288326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc794b6-880b-48aa-8dea-03d3475c8cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:15.289001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc794b6-880b-48aa-8dea-03d3475c8cbc None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.289337 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc794b6-880b-48aa-8dea-03d3475c8cbc None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.289869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc794b6-880b-48aa-8dea-03d3475c8cbc None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.290348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc794b6-880b-48aa-8dea-03d3475c8cbc None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:15.290656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc794b6-880b-48aa-8dea-03d3475c8cbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:15.292156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc794b6-880b-48aa-8dea-03d3475c8cbc None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:15.300534 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 146/276] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:15.302015 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422aaa9-e4b9-4b8c-a1e2-7f31db660d90 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.302312 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422aaa9-e4b9-4b8c-a1e2-7f31db660d90 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.302594 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422aaa9-e4b9-4b8c-a1e2-7f31db660d90 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.303965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 147/277] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:15.308568 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422aaa9-e4b9-4b8c-a1e2-7f31db660d90 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.350846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e290c-59d0-4566-8343-4bd536a97fa1 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:15.385410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50fc41-5577-4b61-8c81-dbdafc436f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:15.386031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50fc41-5577-4b61-8c81-dbdafc436f4f None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.386312 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50fc41-5577-4b61-8c81-dbdafc436f4f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.386587 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50fc41-5577-4b61-8c81-dbdafc436f4f None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.387308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50fc41-5577-4b61-8c81-dbdafc436f4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:15.388766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50fc41-5577-4b61-8c81-dbdafc436f4f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:15.416969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422aaa9-e4b9-4b8c-a1e2-7f31db660d90 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:15.421404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 131/278] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:15 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:28:15.422884 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5980f-e193-4355-9718-3d71009ec442 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.423197 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5980f-e193-4355-9718-3d71009ec442 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.423475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5980f-e193-4355-9718-3d71009ec442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.452752 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-35b5980f-e193-4355-9718-3d71009ec442 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:15.454455 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-35b5980f-e193-4355-9718-3d71009ec442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:15.454798 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9843e-f1bd-4076-b967-e7a4490c3961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:15.455859 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9843e-f1bd-4076-b967-e7a4490c3961 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.456301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9843e-f1bd-4076-b967-e7a4490c3961 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.456760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9843e-f1bd-4076-b967-e7a4490c3961 None admin] PATH_INFO: `/v3/users/2da128e584cc4533ae9dc86b23d1e9c9` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.473452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9843e-f1bd-4076-b967-e7a4490c3961 None admin] RBAC: Authorizing `identity:get_user(user_id=2da128e584cc4533ae9dc86b23d1e9c9)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:15.475258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9843e-f1bd-4076-b967-e7a4490c3961 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:15.482577 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 148/279] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:15 2022] GET /identity/v3/users/2da128e584cc4533ae9dc86b23d1e9c9 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:15.483694 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9843e-f1bd-4076-b967-e7a4490c3961 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.532151 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 132/280] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:15.555660 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5980f-e193-4355-9718-3d71009ec442 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.606976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9843e-f1bd-4076-b967-e7a4490c3961 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:15.649815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:15.650860 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.651339 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.651618 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.652578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] RBAC: Adding query filter params (role.id=251b2e26a5834411bf902ba50b5ec81f, user.id=2da128e584cc4533ae9dc86b23d1e9c9, scope.system=all) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:15.652897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:15.654468 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:15.660378 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 149/281] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:28:15 2022] GET /identity/v3/role_assignments?user.id=2da128e584cc4533ae9dc86b23d1e9c9&scope.system=all&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:15.671829 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5980f-e193-4355-9718-3d71009ec442 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:15.722021 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9eb262-4b23-4780-9027-08d93ee352a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:15.722897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9eb262-4b23-4780-9027-08d93ee352a2 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.723188 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9eb262-4b23-4780-9027-08d93ee352a2 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.723478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9eb262-4b23-4780-9027-08d93ee352a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.738597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9eb262-4b23-4780-9027-08d93ee352a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-cf9eb262-4b23-4780-9027-08d93ee352a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:15.750809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:15.751867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:15.752841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:15.757152 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 133/282] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:15.763094 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.854994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e395c-b70c-45c3-8728-bd2ede35b52d None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.855659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e395c-b70c-45c3-8728-bd2ede35b52d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.856201 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e395c-b70c-45c3-8728-bd2ede35b52d None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.858154 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 134/283] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:15.864263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881517c-c826-40a6-bbb3-d7d2bb6d990c None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.865330 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881517c-c826-40a6-bbb3-d7d2bb6d990c None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.865828 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881517c-c826-40a6-bbb3-d7d2bb6d990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.883021 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd7a0c-fe34-4a2d-bb3f-2771aeaa15c7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:15.887628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-4881517c-c826-40a6-bbb3-d7d2bb6d990c None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:15.889147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-4881517c-c826-40a6-bbb3-d7d2bb6d990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:15.920610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:15.921248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:15.921536 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:15.921815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:15.922592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:15.922908 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:15.924298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:15.937169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 150/284] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:15.938968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:15.965029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 135/285] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:15.995876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881517c-c826-40a6-bbb3-d7d2bb6d990c None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:16.041301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fb265-34d6-4cf0-91a1-13de13b58886 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:16.076419 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d25b5-a3cc-4e20-a7f9-ad4f2a9c009b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:16.077354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d25b5-a3cc-4e20-a7f9-ad4f2a9c009b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:16.077646 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d25b5-a3cc-4e20-a7f9-ad4f2a9c009b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:16.077931 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d25b5-a3cc-4e20-a7f9-ad4f2a9c009b None admin] PATH_INFO: `/v3/users/3943f8b44f12485c88f96089b3c438ef` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:16.079350 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d25b5-a3cc-4e20-a7f9-ad4f2a9c009b None admin] RBAC: Authorizing `identity:get_user(user_id=3943f8b44f12485c88f96089b3c438ef)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:16.080850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d25b5-a3cc-4e20-a7f9-ad4f2a9c009b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:16.092209 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 151/286] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:15 2022] GET /identity/v3/users/3943f8b44f12485c88f96089b3c438ef => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:16.098407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d25b5-a3cc-4e20-a7f9-ad4f2a9c009b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:16.132864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881517c-c826-40a6-bbb3-d7d2bb6d990c None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:16.185713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe039e-3224-4d54-bc48-d2eef30eb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:16.186450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe039e-3224-4d54-bc48-d2eef30eb206 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:16.186738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe039e-3224-4d54-bc48-d2eef30eb206 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:16.187015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe039e-3224-4d54-bc48-d2eef30eb206 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:16.206941 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe039e-3224-4d54-bc48-d2eef30eb206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:16.210277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d25b5-a3cc-4e20-a7f9-ad4f2a9c009b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-33fe039e-3224-4d54-bc48-d2eef30eb206 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:16.218983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:16.221258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:16.223467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 136/287] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:16.229323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe039e-3224-4d54-bc48-d2eef30eb206 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:16.245193 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac203641-b180-4366-93be-da404cf60f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:16.245858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac203641-b180-4366-93be-da404cf60f27 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:16.246147 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac203641-b180-4366-93be-da404cf60f27 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:16.246428 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac203641-b180-4366-93be-da404cf60f27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:16.331302 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac203641-b180-4366-93be-da404cf60f27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:16.332821 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac203641-b180-4366-93be-da404cf60f27 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:16.399599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe039e-3224-4d54-bc48-d2eef30eb206 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:16.450441 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:16.451046 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:16.451332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:16.451614 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:16.452443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:16.452763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:16.454233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:16.468785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 137/288] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:16.474582 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:16.481145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 152/289] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:16.482883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac203641-b180-4366-93be-da404cf60f27 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:16.598642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac203641-b180-4366-93be-da404cf60f27 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:16.634221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a52d-e966-417c-bb50-17b2fc4febbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:16.635090 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a52d-e966-417c-bb50-17b2fc4febbe None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:16.635377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a52d-e966-417c-bb50-17b2fc4febbe None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:16.635656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a52d-e966-417c-bb50-17b2fc4febbe None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:16.636883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9a52d-e966-417c-bb50-17b2fc4febbe None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:16.638369 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9a52d-e966-417c-bb50-17b2fc4febbe None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:16.644971 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 153/290] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:16 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:16.646223 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a52d-e966-417c-bb50-17b2fc4febbe None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:16.692830 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a335d8f-2d92-4f37-bd02-6d82ae989be3 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:16.749548 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a52d-e966-417c-bb50-17b2fc4febbe None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:16.762270 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0cddb-e421-4cda-bcca-045542cd8b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:16.763164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0cddb-e421-4cda-bcca-045542cd8b78 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:16.763451 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0cddb-e421-4cda-bcca-045542cd8b78 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:16.763731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0cddb-e421-4cda-bcca-045542cd8b78 None admin] PATH_INFO: `/v3/users/54f5af49266449b6a81b3ab46af8a1d3` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:16.770367 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0cddb-e421-4cda-bcca-045542cd8b78 None admin] RBAC: Authorizing `identity:get_user(user_id=54f5af49266449b6a81b3ab46af8a1d3)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:16.771859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0cddb-e421-4cda-bcca-045542cd8b78 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:16.773909 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 138/291] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:16 2022] GET /identity/v3/users/54f5af49266449b6a81b3ab46af8a1d3 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:16.778890 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0cddb-e421-4cda-bcca-045542cd8b78 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:16.786437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:16.787107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:16.787396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:16.787679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:16.788537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] RBAC: Adding query filter params (role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=d6cb7aefad52496890bce00f2bb83781, user.id=3943f8b44f12485c88f96089b3c438ef) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:16.788857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:16.790773 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:16.801554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 154/292] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:16 2022] GET /identity/v3/role_assignments?user.id=3943f8b44f12485c88f96089b3c438ef&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:16.954019 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0cddb-e421-4cda-bcca-045542cd8b78 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:17.009951 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4332a-5385-4eeb-b7b0-cca74106250b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:17.010748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4332a-5385-4eeb-b7b0-cca74106250b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:17.011032 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4332a-5385-4eeb-b7b0-cca74106250b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:17.011305 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4332a-5385-4eeb-b7b0-cca74106250b None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:17.017115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4332a-5385-4eeb-b7b0-cca74106250b None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:17.017115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4332a-5385-4eeb-b7b0-cca74106250b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:17.017709 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 139/293] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:16 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:17.023298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:17.146319 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4938d-959b-4490-b5bb-67316ee6df63 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:17.187980 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9d55d-b844-4be5-b825-4e4fa76bbeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:17.189634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9d55d-b844-4be5-b825-4e4fa76bbeb6 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:17.190072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9d55d-b844-4be5-b825-4e4fa76bbeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:17.190490 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9d55d-b844-4be5-b825-4e4fa76bbeb6 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781/users/54f5af49266449b6a81b3ab46af8a1d3/roles/6d053d567af44b6685295deb347f550e` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:17.195317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9d55d-b844-4be5-b825-4e4fa76bbeb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cb7aefad52496890bce00f2bb83781, user_id=54f5af49266449b6a81b3ab46af8a1d3, role_id=6d053d567af44b6685295deb347f550e)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:17.196977 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9d55d-b844-4be5-b825-4e4fa76bbeb6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:17.225529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 155/294] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:17 2022] PUT /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781/users/54f5af49266449b6a81b3ab46af8a1d3/roles/6d053d567af44b6685295deb347f550e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:18.261076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd75148-055f-44fb-808d-c4c573fe6298 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.261076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd75148-055f-44fb-808d-c4c573fe6298 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.261076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd75148-055f-44fb-808d-c4c573fe6298 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.262944 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 140/295] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:18.268561 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54575bb6-369a-4f4c-87e4-83730c32599e None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.268997 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54575bb6-369a-4f4c-87e4-83730c32599e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.269212 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54575bb6-369a-4f4c-87e4-83730c32599e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.286393 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-54575bb6-369a-4f4c-87e4-83730c32599e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:18.287950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-54575bb6-369a-4f4c-87e4-83730c32599e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:18.320976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e67af5-24c2-448b-ae0c-023e6e06431c None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.321656 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e67af5-24c2-448b-ae0c-023e6e06431c None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.322164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e67af5-24c2-448b-ae0c-023e6e06431c None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.323997 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 141/296] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:18.329165 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a01625-f4ea-4029-b1e9-8aed33ecfdc3 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.329710 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a01625-f4ea-4029-b1e9-8aed33ecfdc3 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.330219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a01625-f4ea-4029-b1e9-8aed33ecfdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.347206 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-c7a01625-f4ea-4029-b1e9-8aed33ecfdc3 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:18.348823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-c7a01625-f4ea-4029-b1e9-8aed33ecfdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:18.357717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 156/297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:18.380354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91c36a-9857-43ab-b719-94b190c08998 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.380920 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91c36a-9857-43ab-b719-94b190c08998 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.381240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91c36a-9857-43ab-b719-94b190c08998 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.382803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 157/298] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:18.383358 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91c36a-9857-43ab-b719-94b190c08998 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:18.429071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 142/299] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:18.429071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2fce7-f4f4-480f-bbda-949db57f717f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.429071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2fce7-f4f4-480f-bbda-949db57f717f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.429071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2fce7-f4f4-480f-bbda-949db57f717f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.439579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-19e2fce7-f4f4-480f-bbda-949db57f717f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:18.440764 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-19e2fce7-f4f4-480f-bbda-949db57f717f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:18.493249 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91c36a-9857-43ab-b719-94b190c08998 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:18.508077 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 143/300] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:18.509239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2fce7-f4f4-480f-bbda-949db57f717f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:18.529546 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f898e7c-cd2c-4515-8312-ce7bc9ad3e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:18.530338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f898e7c-cd2c-4515-8312-ce7bc9ad3e84 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.530929 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f898e7c-cd2c-4515-8312-ce7bc9ad3e84 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.531216 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f898e7c-cd2c-4515-8312-ce7bc9ad3e84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.532405 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f898e7c-cd2c-4515-8312-ce7bc9ad3e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:18.533892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f898e7c-cd2c-4515-8312-ce7bc9ad3e84 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:18.535678 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 158/301] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:18.536308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f898e7c-cd2c-4515-8312-ce7bc9ad3e84 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:18.623348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2fce7-f4f4-480f-bbda-949db57f717f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:18.638453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f898e7c-cd2c-4515-8312-ce7bc9ad3e84 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:18.658548 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde0e29-83f9-4600-92ad-ffda6ef0d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:18.659582 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde0e29-83f9-4600-92ad-ffda6ef0d7dd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.660593 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde0e29-83f9-4600-92ad-ffda6ef0d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.660874 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde0e29-83f9-4600-92ad-ffda6ef0d7dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.671098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde0e29-83f9-4600-92ad-ffda6ef0d7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:18.673225 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd433b-aa8c-4e29-b336-bfeffcd5cdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:18.674026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd433b-aa8c-4e29-b336-bfeffcd5cdc1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.674310 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd433b-aa8c-4e29-b336-bfeffcd5cdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.674590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd433b-aa8c-4e29-b336-bfeffcd5cdc1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-cbde0e29-83f9-4600-92ad-ffda6ef0d7dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:18.679508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:18.680497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 144/302] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:18.681885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde0e29-83f9-4600-92ad-ffda6ef0d7dd None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:18.691329 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd433b-aa8c-4e29-b336-bfeffcd5cdc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-f6fd433b-aa8c-4e29-b336-bfeffcd5cdc1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:18.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:18.701059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:18.701958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:18.707581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 159/303] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:18.707581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd433b-aa8c-4e29-b336-bfeffcd5cdc1 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:18.782659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde0e29-83f9-4600-92ad-ffda6ef0d7dd None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:18.803955 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd433b-aa8c-4e29-b336-bfeffcd5cdc1 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:18.816568 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38135cf-f2da-494c-828c-060cf8125665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:18.817458 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38135cf-f2da-494c-828c-060cf8125665 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.817927 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38135cf-f2da-494c-828c-060cf8125665 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.818359 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38135cf-f2da-494c-828c-060cf8125665 None admin] PATH_INFO: `/v3/groups` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.819209 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38135cf-f2da-494c-828c-060cf8125665 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:18.820820 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38135cf-f2da-494c-828c-060cf8125665 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:18.836918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:18.837968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:18.838430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:18.838603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 145/304] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 9 15:28:18 2022] POST /identity/v3/groups => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:18.839136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:18.839453 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38135cf-f2da-494c-828c-060cf8125665 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:18.840290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:18.840609 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:18.842095 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:18.858540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 160/305] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:18.866063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:18.988897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17196ae5-fe3b-43eb-8a77-0c55d434b383 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:19.000330 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38135cf-f2da-494c-828c-060cf8125665 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:19.036934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a607841-035b-4343-aa1c-aaac0b5f5f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:19.037183 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:19.038010 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:19.038373 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:19.038724 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a607841-035b-4343-aa1c-aaac0b5f5f48 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:19.038852 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:19.039336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a607841-035b-4343-aa1c-aaac0b5f5f48 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:19.039666 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:19.040057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:19.040340 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a607841-035b-4343-aa1c-aaac0b5f5f48 None admin] PATH_INFO: `/v3/users/2da128e584cc4533ae9dc86b23d1e9c9` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:19.041788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:19.042067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a607841-035b-4343-aa1c-aaac0b5f5f48 None admin] RBAC: Authorizing `identity:get_user(user_id=2da128e584cc4533ae9dc86b23d1e9c9)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:19.044001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a607841-035b-4343-aa1c-aaac0b5f5f48 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:19.046663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 161/306] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:18 2022] GET /identity/v3/users/2da128e584cc4533ae9dc86b23d1e9c9 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:19.053508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a607841-035b-4343-aa1c-aaac0b5f5f48 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:19.055977 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 146/307] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:19.062114 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:19.180333 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a607841-035b-4343-aa1c-aaac0b5f5f48 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:19.199750 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972dad-3938-440e-b244-89f7fc2310ad None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:19.227599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755b88-7c98-4488-9ffa-81606462abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:19.229327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755b88-7c98-4488-9ffa-81606462abed None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:19.230108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755b88-7c98-4488-9ffa-81606462abed None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:19.230658 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755b88-7c98-4488-9ffa-81606462abed None admin] PATH_INFO: `/v3/system/users/2da128e584cc4533ae9dc86b23d1e9c9/roles/251b2e26a5834411bf902ba50b5ec81f` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:19.233442 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75755b88-7c98-4488-9ffa-81606462abed None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2da128e584cc4533ae9dc86b23d1e9c9, role_id=251b2e26a5834411bf902ba50b5ec81f)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:19.235855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75755b88-7c98-4488-9ffa-81606462abed None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:19.239757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eccde-558a-46ca-a472-2cc472f03b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:19.241354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eccde-558a-46ca-a472-2cc472f03b93 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:19.241645 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eccde-558a-46ca-a472-2cc472f03b93 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:19.241938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eccde-558a-46ca-a472-2cc472f03b93 None admin] PATH_INFO: `/v3/users/9a357d94a1a94d738b4a4f31c5dc680a` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:19.248915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 162/308] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 9 15:28:19 2022] PUT /identity/v3/system/users/2da128e584cc4533ae9dc86b23d1e9c9/roles/251b2e26a5834411bf902ba50b5ec81f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:19.261472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5eccde-558a-46ca-a472-2cc472f03b93 None admin] RBAC: Authorizing `identity:get_user(user_id=9a357d94a1a94d738b4a4f31c5dc680a)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:19.262999 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5eccde-558a-46ca-a472-2cc472f03b93 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:19.265275 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 147/309] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:19 2022] GET /identity/v3/users/9a357d94a1a94d738b4a4f31c5dc680a => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:19.272144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755b88-7c98-4488-9ffa-81606462abed None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:19.386006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755b88-7c98-4488-9ffa-81606462abed None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:19.423512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac08360-0182-447d-aa42-efcd362016d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:19.424603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac08360-0182-447d-aa42-efcd362016d1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:19.425138 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac08360-0182-447d-aa42-efcd362016d1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:19.425676 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac08360-0182-447d-aa42-efcd362016d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:19.426772 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac08360-0182-447d-aa42-efcd362016d1 None admin] RBAC: Adding query filter params (role.id=6d053d567af44b6685295deb347f550e, user.id=9a357d94a1a94d738b4a4f31c5dc680a, scope.system=all) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:19.427305 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac08360-0182-447d-aa42-efcd362016d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:19.429006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac08360-0182-447d-aa42-efcd362016d1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:19.438169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 163/310] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:28:19 2022] GET /identity/v3/role_assignments?user.id=9a357d94a1a94d738b4a4f31c5dc680a&scope.system=all&role.id=6d053d567af44b6685295deb347f550e => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:19.714333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12562c6-570d-420b-8ef3-cc20c4f00dbc None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:19.714724 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12562c6-570d-420b-8ef3-cc20c4f00dbc None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:19.714932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12562c6-570d-420b-8ef3-cc20c4f00dbc None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:19.716505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 148/311] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:19.722139 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611924d7-7f62-4801-afd0-b69a4379bb83 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:19.722436 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611924d7-7f62-4801-afd0-b69a4379bb83 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:19.722717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611924d7-7f62-4801-afd0-b69a4379bb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:19.740258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-611924d7-7f62-4801-afd0-b69a4379bb83 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:19.741483 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-611924d7-7f62-4801-afd0-b69a4379bb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:19.814922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 164/312] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:19.837855 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12562c6-570d-420b-8ef3-cc20c4f00dbc None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:19.940240 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12562c6-570d-420b-8ef3-cc20c4f00dbc None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:19.974460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c39b-ee17-48a0-9b29-84cb25712307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:19.975173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c39b-ee17-48a0-9b29-84cb25712307 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:19.975459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c39b-ee17-48a0-9b29-84cb25712307 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:19.975734 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c39b-ee17-48a0-9b29-84cb25712307 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:19.986129 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66c39b-ee17-48a0-9b29-84cb25712307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-9f66c39b-ee17-48a0-9b29-84cb25712307 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:19.994268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:19.995293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:19.996608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 149/313] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:20.001579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611924d7-7f62-4801-afd0-b69a4379bb83 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:20.106520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611924d7-7f62-4801-afd0-b69a4379bb83 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:20.142063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:20.142732 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:20.143013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:20.143286 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:20.144038 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] RBAC: Adding query filter params (name=reader) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:20.144388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:20.145805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:20.154016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 165/314] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:20.159486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c39b-ee17-48a0-9b29-84cb25712307 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:20.261631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c39b-ee17-48a0-9b29-84cb25712307 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:20.295321 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f4d50-93c6-487c-baca-c6c79119b84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:20.296193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f4d50-93c6-487c-baca-c6c79119b84b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:20.296473 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f4d50-93c6-487c-baca-c6c79119b84b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:20.296745 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f4d50-93c6-487c-baca-c6c79119b84b None admin] PATH_INFO: `/v3/users/54f5af49266449b6a81b3ab46af8a1d3` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:20.297979 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f4d50-93c6-487c-baca-c6c79119b84b None admin] RBAC: Authorizing `identity:get_user(user_id=54f5af49266449b6a81b3ab46af8a1d3)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:20.299601 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f4d50-93c6-487c-baca-c6c79119b84b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:20.302271 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 150/315] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:20 2022] GET /identity/v3/users/54f5af49266449b6a81b3ab46af8a1d3 => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:20.307723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:20.409387 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e6f1b-a8c2-4d0c-8e84-736719074874 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:20.443183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1cd2a-b86c-4083-98cb-f434b3919f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:20.443835 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1cd2a-b86c-4083-98cb-f434b3919f8b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:20.444118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1cd2a-b86c-4083-98cb-f434b3919f8b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:20.444396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1cd2a-b86c-4083-98cb-f434b3919f8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:20.512475 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1cd2a-b86c-4083-98cb-f434b3919f8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:20.513882 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1cd2a-b86c-4083-98cb-f434b3919f8b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:20.612567 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 166/316] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:20.617929 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f4d50-93c6-487c-baca-c6c79119b84b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:20.720021 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f4d50-93c6-487c-baca-c6c79119b84b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:20.754215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935703c3-7b6c-4a33-9a52-bdcc99306462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:20.755083 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935703c3-7b6c-4a33-9a52-bdcc99306462 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:20.755363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935703c3-7b6c-4a33-9a52-bdcc99306462 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:20.755637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935703c3-7b6c-4a33-9a52-bdcc99306462 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:20.756833 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935703c3-7b6c-4a33-9a52-bdcc99306462 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:20.758306 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935703c3-7b6c-4a33-9a52-bdcc99306462 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:20.760303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 151/317] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:20 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:20.766142 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1cd2a-b86c-4083-98cb-f434b3919f8b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:20.870384 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1cd2a-b86c-4083-98cb-f434b3919f8b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:20.904999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5ad47-422a-4910-931d-9e921bc9e73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:20.905773 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5ad47-422a-4910-931d-9e921bc9e73d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:20.906041 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5ad47-422a-4910-931d-9e921bc9e73d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:20.906316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5ad47-422a-4910-931d-9e921bc9e73d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:20.907156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5ad47-422a-4910-931d-9e921bc9e73d None admin] RBAC: Adding query filter params (role.id=6d053d567af44b6685295deb347f550e, scope.project.id=d6cb7aefad52496890bce00f2bb83781, user.id=54f5af49266449b6a81b3ab46af8a1d3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:20.907471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5ad47-422a-4910-931d-9e921bc9e73d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:20.908902 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5ad47-422a-4910-931d-9e921bc9e73d None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:20.914799 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 167/318] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:20 2022] GET /identity/v3/role_assignments?user.id=54f5af49266449b6a81b3ab46af8a1d3&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=6d053d567af44b6685295deb347f550e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:21.248064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a021-e10d-4018-aa51-087ee6c787c8 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.248446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a021-e10d-4018-aa51-087ee6c787c8 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.248647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a021-e10d-4018-aa51-087ee6c787c8 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.250232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 152/319] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:21.256017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90def41a-8929-4bcf-8415-15e1d85d28d4 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.256344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90def41a-8929-4bcf-8415-15e1d85d28d4 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.256626 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90def41a-8929-4bcf-8415-15e1d85d28d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.273172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-90def41a-8929-4bcf-8415-15e1d85d28d4 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:21.274376 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-90def41a-8929-4bcf-8415-15e1d85d28d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:21.338545 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 168/320] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:21.361679 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a021-e10d-4018-aa51-087ee6c787c8 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:21.466303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a021-e10d-4018-aa51-087ee6c787c8 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:21.502291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ba1ef-3c91-4edc-89d3-e0825a6e84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:21.505543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba1ef-3c91-4edc-89d3-e0825a6e84b9 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.505543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba1ef-3c91-4edc-89d3-e0825a6e84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.505543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba1ef-3c91-4edc-89d3-e0825a6e84b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.506661 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ba1ef-3c91-4edc-89d3-e0825a6e84b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:21.508257 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ba1ef-3c91-4edc-89d3-e0825a6e84b9 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:21.510166 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 153/321] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:21.515735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90def41a-8929-4bcf-8415-15e1d85d28d4 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:21.546147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056a23-6f19-4829-abf1-c8fce437abcc None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.546440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056a23-6f19-4829-abf1-c8fce437abcc None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.546715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056a23-6f19-4829-abf1-c8fce437abcc None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.548198 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 154/322] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:21.553498 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cac7d-1d69-4e5e-a319-e4beece7b106 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.553791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cac7d-1d69-4e5e-a319-e4beece7b106 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.554073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129cac7d-1d69-4e5e-a319-e4beece7b106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.572094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-129cac7d-1d69-4e5e-a319-e4beece7b106 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:21.573643 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-129cac7d-1d69-4e5e-a319-e4beece7b106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:21.620458 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90def41a-8929-4bcf-8415-15e1d85d28d4 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:21.644691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 155/323] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:21.657922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6daa2-4dfc-4165-ace8-129f6e763cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:21.658596 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6daa2-4dfc-4165-ace8-129f6e763cd7 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.658884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6daa2-4dfc-4165-ace8-129f6e763cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.659162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6daa2-4dfc-4165-ace8-129f6e763cd7 None admin] PATH_INFO: `/v3/groups` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.660404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6daa2-4dfc-4165-ace8-129f6e763cd7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:21.661894 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6daa2-4dfc-4165-ace8-129f6e763cd7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:21.667855 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cac7d-1d69-4e5e-a319-e4beece7b106 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:21.680480 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 169/324] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 9 15:28:21 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:21.771363 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26184846-cef0-40a0-a441-ca8d02f46b51 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.771661 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26184846-cef0-40a0-a441-ca8d02f46b51 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.771946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26184846-cef0-40a0-a441-ca8d02f46b51 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.773552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 170/325] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:21.779271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5acbbc-9e60-4a86-90ce-3252486c2f77 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.779585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5acbbc-9e60-4a86-90ce-3252486c2f77 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.779828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5acbbc-9e60-4a86-90ce-3252486c2f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.783910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129cac7d-1d69-4e5e-a319-e4beece7b106 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:21.804159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-7f5acbbc-9e60-4a86-90ce-3252486c2f77 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:21.805532 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-7f5acbbc-9e60-4a86-90ce-3252486c2f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:21.825031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c570dca-6035-4a2c-858d-c3c8212305e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:21.825903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c570dca-6035-4a2c-858d-c3c8212305e6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.826191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c570dca-6035-4a2c-858d-c3c8212305e6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.826471 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c570dca-6035-4a2c-858d-c3c8212305e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.837726 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c570dca-6035-4a2c-858d-c3c8212305e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-4c570dca-6035-4a2c-858d-c3c8212305e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:21.845632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:21.846691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:21.850045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 156/326] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:21.853577 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c570dca-6035-4a2c-858d-c3c8212305e6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:21.882392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 171/327] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:21.906224 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5acbbc-9e60-4a86-90ce-3252486c2f77 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:21.960757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c570dca-6035-4a2c-858d-c3c8212305e6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:21.997142 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:21.997831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:21.998117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:21.998399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:21.999162 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] RBAC: Adding query filter params (name=member) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:21.999476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:22.000856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:22.009895 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 157/328] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:22.016847 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:22.027836 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5acbbc-9e60-4a86-90ce-3252486c2f77 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:22.069081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fde318-3119-4a1d-8744-f2d5067ddc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:22.070287 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fde318-3119-4a1d-8744-f2d5067ddc2c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:22.070711 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fde318-3119-4a1d-8744-f2d5067ddc2c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:22.071125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fde318-3119-4a1d-8744-f2d5067ddc2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:22.089325 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fde318-3119-4a1d-8744-f2d5067ddc2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-07fde318-3119-4a1d-8744-f2d5067ddc2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:22.111430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:22.113474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:22.115040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 172/329] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:22.118729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fde318-3119-4a1d-8744-f2d5067ddc2c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:22.137208 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e467b8-57f2-42d9-9562-f89ed30426af None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:22.179826 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca6991-ff0d-4713-9727-59c350d88203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:22.180791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6991-ff0d-4713-9727-59c350d88203 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:22.181209 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6991-ff0d-4713-9727-59c350d88203 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:22.181483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6991-ff0d-4713-9727-59c350d88203 None admin] PATH_INFO: `/v3/users/2da128e584cc4533ae9dc86b23d1e9c9` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:22.182812 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca6991-ff0d-4713-9727-59c350d88203 None admin] RBAC: Authorizing `identity:get_user(user_id=2da128e584cc4533ae9dc86b23d1e9c9)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:22.184198 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca6991-ff0d-4713-9727-59c350d88203 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:22.186253 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 158/330] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:22 2022] GET /identity/v3/users/2da128e584cc4533ae9dc86b23d1e9c9 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:22.192023 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca6991-ff0d-4713-9727-59c350d88203 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:22.239580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fde318-3119-4a1d-8744-f2d5067ddc2c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:22.274699 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:22.275747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:22.276235 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:22.276754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:22.277758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:22.278274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:22.279852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:22.289671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 173/331] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:22.297063 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca6991-ff0d-4713-9727-59c350d88203 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:22.298285 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:22.333493 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:22.334215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:22.334505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:22.334782 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:22.335634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] RBAC: Adding query filter params (role.id=251b2e26a5834411bf902ba50b5ec81f, user.id=2da128e584cc4533ae9dc86b23d1e9c9, scope.system=all) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:22.335947 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:22.337427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:22.350400 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 159/332] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:28:22 2022] GET /identity/v3/role_assignments?user.id=2da128e584cc4533ae9dc86b23d1e9c9&scope.system=all&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:22.409289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384d142-508d-495b-8d88-44d3a482e1b4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:22.447411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990d920-4259-4c24-9dbc-17c8c5f24608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:22.448324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990d920-4259-4c24-9dbc-17c8c5f24608 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:22.448616 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990d920-4259-4c24-9dbc-17c8c5f24608 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:22.448898 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990d920-4259-4c24-9dbc-17c8c5f24608 None admin] PATH_INFO: `/v3/users/9a357d94a1a94d738b4a4f31c5dc680a` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:22.450955 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990d920-4259-4c24-9dbc-17c8c5f24608 None admin] RBAC: Authorizing `identity:get_user(user_id=9a357d94a1a94d738b4a4f31c5dc680a)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:22.453728 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990d920-4259-4c24-9dbc-17c8c5f24608 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:22.458068 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 174/333] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:22 2022] GET /identity/v3/users/9a357d94a1a94d738b4a4f31c5dc680a => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:22.463636 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:22.578821 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18622a-cd08-4bbb-8391-262ed8229fb0 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:22.613366 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfdf1b-ebb4-4e1a-8c31-9a0d458fd314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:22.614247 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfdf1b-ebb4-4e1a-8c31-9a0d458fd314 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:22.614531 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfdf1b-ebb4-4e1a-8c31-9a0d458fd314 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:22.614809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfdf1b-ebb4-4e1a-8c31-9a0d458fd314 None admin] PATH_INFO: `/v3/system/users/9a357d94a1a94d738b4a4f31c5dc680a/roles/6d053d567af44b6685295deb347f550e` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:22.616355 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfdf1b-ebb4-4e1a-8c31-9a0d458fd314 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9a357d94a1a94d738b4a4f31c5dc680a, role_id=6d053d567af44b6685295deb347f550e)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:22.617797 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfdf1b-ebb4-4e1a-8c31-9a0d458fd314 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:22.630216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 160/334] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 9 15:28:22 2022] PUT /identity/v3/system/users/9a357d94a1a94d738b4a4f31c5dc680a/roles/6d053d567af44b6685295deb347f550e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:24.383297 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5043-c3f3-4e81-9725-bb9a84f57dba None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:24.383804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5043-c3f3-4e81-9725-bb9a84f57dba None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:24.383804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe5043-c3f3-4e81-9725-bb9a84f57dba None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:24.385320 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 175/335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:24.390904 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e47de-4e88-4ef1-932e-c08cee8336fe None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:24.391323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e47de-4e88-4ef1-932e-c08cee8336fe None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:24.391516 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e47de-4e88-4ef1-932e-c08cee8336fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:24.414059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-f40e47de-4e88-4ef1-932e-c08cee8336fe None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:24.415311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-f40e47de-4e88-4ef1-932e-c08cee8336fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:24.493635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 161/336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:24.529299 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5043-c3f3-4e81-9725-bb9a84f57dba None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:24.564561 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159b59d-af18-4802-bfad-d054b24b3fa6 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:24.564903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159b59d-af18-4802-bfad-d054b24b3fa6 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:24.565879 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159b59d-af18-4802-bfad-d054b24b3fa6 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:24.567999 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 162/337] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:24.582585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cdacc-b820-4200-8e12-735fadf55703 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:24.583195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cdacc-b820-4200-8e12-735fadf55703 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:24.583678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cdacc-b820-4200-8e12-735fadf55703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:24.613772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-c45cdacc-b820-4200-8e12-735fadf55703 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:24.615309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-c45cdacc-b820-4200-8e12-735fadf55703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:24.663835 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe5043-c3f3-4e81-9725-bb9a84f57dba None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:24.708930 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb663a-e895-480f-984c-aed2461817c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:24.715759 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb663a-e895-480f-984c-aed2461817c7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:24.716323 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb663a-e895-480f-984c-aed2461817c7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:24.716811 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb663a-e895-480f-984c-aed2461817c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:24.745506 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb663a-e895-480f-984c-aed2461817c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-dbdb663a-e895-480f-984c-aed2461817c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:24.757893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:24.763380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:24.764487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 176/338] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:24.769878 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb663a-e895-480f-984c-aed2461817c7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:24.780275 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 163/339] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:24.840239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cdacc-b820-4200-8e12-735fadf55703 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:24.952479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb663a-e895-480f-984c-aed2461817c7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.009391 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45cdacc-b820-4200-8e12-735fadf55703 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.017144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.017837 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.018121 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.018397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.019830 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:25.020348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:25.024445 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:25.043471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 177/340] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:25.045023 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:25.061950 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec01d-b56a-46ce-bd1d-a8c4d7ff8137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.063176 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec01d-b56a-46ce-bd1d-a8c4d7ff8137 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.063671 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec01d-b56a-46ce-bd1d-a8c4d7ff8137 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.064175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec01d-b56a-46ce-bd1d-a8c4d7ff8137 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.075347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ec01d-b56a-46ce-bd1d-a8c4d7ff8137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-f86ec01d-b56a-46ce-bd1d-a8c4d7ff8137 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:25.089651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:25.090698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:25.091559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:25.100421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 164/341] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:25.108219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec01d-b56a-46ce-bd1d-a8c4d7ff8137 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:25.170338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91316e4c-0900-4672-bd80-67b8c304097b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.225976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5306e-8e6a-4788-9d61-1cf9ac4a40f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.226993 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5306e-8e6a-4788-9d61-1cf9ac4a40f7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.227486 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5306e-8e6a-4788-9d61-1cf9ac4a40f7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.227962 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5306e-8e6a-4788-9d61-1cf9ac4a40f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.259486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec01d-b56a-46ce-bd1d-a8c4d7ff8137 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.302669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.303633 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.304098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.304542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.305491 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:25.305970 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:25.307515 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:25.308747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5306e-8e6a-4788-9d61-1cf9ac4a40f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:25.310425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5306e-8e6a-4788-9d61-1cf9ac4a40f7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:25.317107 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 165/342] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:25.342693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:25.416918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 178/343] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:28:25.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5306e-8e6a-4788-9d61-1cf9ac4a40f7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:25.465741 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248011d1-2a91-48f0-9b32-7c022c30d850 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.500392 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea0698-41fc-484a-a10b-1dbc5a886256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.501194 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea0698-41fc-484a-a10b-1dbc5a886256 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.501587 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea0698-41fc-484a-a10b-1dbc5a886256 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.501972 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea0698-41fc-484a-a10b-1dbc5a886256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.569705 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5306e-8e6a-4788-9d61-1cf9ac4a40f7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.587339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea0698-41fc-484a-a10b-1dbc5a886256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:25.589225 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea0698-41fc-484a-a10b-1dbc5a886256 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:25.622173 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e4976-f471-44a3-b6e4-fd9fab3064b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.623386 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4976-f471-44a3-b6e4-fd9fab3064b6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.623831 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4976-f471-44a3-b6e4-fd9fab3064b6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.624266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4976-f471-44a3-b6e4-fd9fab3064b6 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.625746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e4976-f471-44a3-b6e4-fd9fab3064b6 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:25.627365 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e4976-f471-44a3-b6e4-fd9fab3064b6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:25.630183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 179/344] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:25 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:25.638088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e4976-f471-44a3-b6e4-fd9fab3064b6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:25.695821 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 166/345] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:25.705397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea0698-41fc-484a-a10b-1dbc5a886256 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:25.758538 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e4976-f471-44a3-b6e4-fd9fab3064b6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.800183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71176c9-4b6b-4033-96ac-24a49fb329b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.801972 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71176c9-4b6b-4033-96ac-24a49fb329b1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.802486 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71176c9-4b6b-4033-96ac-24a49fb329b1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.802993 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71176c9-4b6b-4033-96ac-24a49fb329b1 None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.817763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71176c9-4b6b-4033-96ac-24a49fb329b1 None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:25.819632 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71176c9-4b6b-4033-96ac-24a49fb329b1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:25.821615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 180/346] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:25 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:25.827427 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71176c9-4b6b-4033-96ac-24a49fb329b1 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:25.830057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea0698-41fc-484a-a10b-1dbc5a886256 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.869084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f057c9c-40da-4735-a54a-ae6c50891b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.869084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f057c9c-40da-4735-a54a-ae6c50891b48 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.869084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f057c9c-40da-4735-a54a-ae6c50891b48 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.869084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f057c9c-40da-4735-a54a-ae6c50891b48 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.869534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f057c9c-40da-4735-a54a-ae6c50891b48 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:25.870712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f057c9c-40da-4735-a54a-ae6c50891b48 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:25.872720 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 167/347] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:25 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:25.878361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f057c9c-40da-4735-a54a-ae6c50891b48 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:25.947456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71176c9-4b6b-4033-96ac-24a49fb329b1 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:25.989735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8eea5f-cb2a-4e5c-83b3-937e0d714f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:25.990918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8eea5f-cb2a-4e5c-83b3-937e0d714f68 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:25.991433 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8eea5f-cb2a-4e5c-83b3-937e0d714f68 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:25.991923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8eea5f-cb2a-4e5c-83b3-937e0d714f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:25.992991 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8eea5f-cb2a-4e5c-83b3-937e0d714f68 None admin] RBAC: Adding query filter params (group.id=5e27e058f298484bbc650a1b2c1ca9fc, role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=3d209aef7bbd4477a103fd4064afe530) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:25.993557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8eea5f-cb2a-4e5c-83b3-937e0d714f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:25.995236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8eea5f-cb2a-4e5c-83b3-937e0d714f68 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:26.002524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 181/348] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:25 2022] GET /identity/v3/role_assignments?group.id=5e27e058f298484bbc650a1b2c1ca9fc&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:26.006010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e76e3-62c2-4de7-ae7f-022afe5fdef8 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.006577 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e76e3-62c2-4de7-ae7f-022afe5fdef8 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.007110 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e76e3-62c2-4de7-ae7f-022afe5fdef8 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.008940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 182/349] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:26.012580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f057c9c-40da-4735-a54a-ae6c50891b48 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:26.014199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c118-7c82-4f5e-a38e-715a1fa4b7a0 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.014716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c118-7c82-4f5e-a38e-715a1fa4b7a0 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.015210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c118-7c82-4f5e-a38e-715a1fa4b7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.045640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-eb84c118-7c82-4f5e-a38e-715a1fa4b7a0 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:26.047255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-eb84c118-7c82-4f5e-a38e-715a1fa4b7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:26.049849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab93b0-8693-4942-9ef8-3fd15af7a83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:26.052004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab93b0-8693-4942-9ef8-3fd15af7a83c None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.052004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab93b0-8693-4942-9ef8-3fd15af7a83c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.052004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab93b0-8693-4942-9ef8-3fd15af7a83c None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.052998 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab93b0-8693-4942-9ef8-3fd15af7a83c None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:26.055590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab93b0-8693-4942-9ef8-3fd15af7a83c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:26.058178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 168/350] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:25 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:26.069012 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab93b0-8693-4942-9ef8-3fd15af7a83c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:26.142595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 183/351] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:26.179456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c118-7c82-4f5e-a38e-715a1fa4b7a0 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:26.193672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab93b0-8693-4942-9ef8-3fd15af7a83c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:26.235548 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb6507-51ba-4ca0-9231-4102982075e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:26.236223 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb6507-51ba-4ca0-9231-4102982075e8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.236508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb6507-51ba-4ca0-9231-4102982075e8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.236786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb6507-51ba-4ca0-9231-4102982075e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.237667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb6507-51ba-4ca0-9231-4102982075e8 None admin] RBAC: Adding query filter params (group.id=5e27e058f298484bbc650a1b2c1ca9fc, role.id=02f09975329342eead41c01b150325c0, scope.project.id=3d209aef7bbd4477a103fd4064afe530) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:26.238556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb6507-51ba-4ca0-9231-4102982075e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:26.240006 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb6507-51ba-4ca0-9231-4102982075e8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:26.257472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 169/352] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:26 2022] GET /identity/v3/role_assignments?group.id=5e27e058f298484bbc650a1b2c1ca9fc&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=02f09975329342eead41c01b150325c0 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:26.320331 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c118-7c82-4f5e-a38e-715a1fa4b7a0 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:26.355428 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbac8ee-77ec-40ba-b586-d2bf6933f3f8 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.355759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbac8ee-77ec-40ba-b586-d2bf6933f3f8 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.355998 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbac8ee-77ec-40ba-b586-d2bf6933f3f8 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.357932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 170/353] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:26.364438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65266a7-0ce3-4715-ada6-6170223b71ef None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.365136 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65266a7-0ce3-4715-ada6-6170223b71ef None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.365644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65266a7-0ce3-4715-ada6-6170223b71ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.384519 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf29d0-93f1-404a-b49a-9421cc4a7b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:26.385389 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf29d0-93f1-404a-b49a-9421cc4a7b22 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.385678 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf29d0-93f1-404a-b49a-9421cc4a7b22 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.385958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf29d0-93f1-404a-b49a-9421cc4a7b22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.401876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-d65266a7-0ce3-4715-ada6-6170223b71ef None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:26.404343 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-d65266a7-0ce3-4715-ada6-6170223b71ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:26.412824 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf29d0-93f1-404a-b49a-9421cc4a7b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-3caf29d0-93f1-404a-b49a-9421cc4a7b22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:26.422825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:26.424324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:28:26.425228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:26.428651 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 184/354] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:26.434958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf29d0-93f1-404a-b49a-9421cc4a7b22 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:26.540017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 171/355] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:26.571229 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65266a7-0ce3-4715-ada6-6170223b71ef None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:26.613722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf29d0-93f1-404a-b49a-9421cc4a7b22 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:26.690519 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:26.691253 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.691539 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.691816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.693629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:26.693946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:26.696210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:26.713634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 185/356] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:26.726182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:26.761396 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65266a7-0ce3-4715-ada6-6170223b71ef None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:26.812568 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f320-7740-42cf-aea6-706087d9fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:26.813447 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f320-7740-42cf-aea6-706087d9fa71 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.813734 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f320-7740-42cf-aea6-706087d9fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.814012 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f320-7740-42cf-aea6-706087d9fa71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.830110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5f320-7740-42cf-aea6-706087d9fa71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-15f5f320-7740-42cf-aea6-706087d9fa71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:26.842687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:26.843761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:26.844641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 172/357] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:26.853117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f320-7740-42cf-aea6-706087d9fa71 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:26.913724 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc75d1-53f3-4315-8e7f-1289416b1b9e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:26.986738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f320-7740-42cf-aea6-706087d9fa71 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:26.993796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94716449-f529-46e6-8de5-5a7f0d15fd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:26.994703 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716449-f529-46e6-8de5-5a7f0d15fd17 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:26.994993 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716449-f529-46e6-8de5-5a7f0d15fd17 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:26.995275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716449-f529-46e6-8de5-5a7f0d15fd17 None admin] PATH_INFO: `/v3/users/9a357d94a1a94d738b4a4f31c5dc680a` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:26.996593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94716449-f529-46e6-8de5-5a7f0d15fd17 None admin] RBAC: Authorizing `identity:get_user(user_id=9a357d94a1a94d738b4a4f31c5dc680a)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:26.998088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94716449-f529-46e6-8de5-5a7f0d15fd17 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:27.000067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 186/358] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:28:26 2022] GET /identity/v3/users/9a357d94a1a94d738b4a4f31c5dc680a => generated 293 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:27.026100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94716449-f529-46e6-8de5-5a7f0d15fd17 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:27.039003 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a99af4-2842-4649-bbf6-12f0d8ac5fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:27.039686 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99af4-2842-4649-bbf6-12f0d8ac5fae None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.039971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99af4-2842-4649-bbf6-12f0d8ac5fae None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.040250 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99af4-2842-4649-bbf6-12f0d8ac5fae None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.041014 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a99af4-2842-4649-bbf6-12f0d8ac5fae None admin] RBAC: Adding query filter params (name=member) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:27.041511 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a99af4-2842-4649-bbf6-12f0d8ac5fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:27.042727 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a99af4-2842-4649-bbf6-12f0d8ac5fae None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:27.051855 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 173/359] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:27.053207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3721759-ad14-4973-81a2-993e573e4a1a None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.053506 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3721759-ad14-4973-81a2-993e573e4a1a None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.053781 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3721759-ad14-4973-81a2-993e573e4a1a None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.055045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 174/360] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:27.061937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3721759-ad14-4973-81a2-993e573e4a1a None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:27.189394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94716449-f529-46e6-8de5-5a7f0d15fd17 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:27.202580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3721759-ad14-4973-81a2-993e573e4a1a None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:27.239278 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ba8bd-a90f-4c31-b002-35eba9c09d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:27.239874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ba8bd-a90f-4c31-b002-35eba9c09d30 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.240157 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ba8bd-a90f-4c31-b002-35eba9c09d30 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.240435 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ba8bd-a90f-4c31-b002-35eba9c09d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.241309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ba8bd-a90f-4c31-b002-35eba9c09d30 None admin] RBAC: Adding query filter params (role.id=6d053d567af44b6685295deb347f550e, user.id=9a357d94a1a94d738b4a4f31c5dc680a, scope.system=all) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:27.241624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ba8bd-a90f-4c31-b002-35eba9c09d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:27.243075 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ba8bd-a90f-4c31-b002-35eba9c09d30 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:27.256347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47191141-cd91-47df-a1e1-c78556275743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:27.257443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47191141-cd91-47df-a1e1-c78556275743 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.257814 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 187/361] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:28:27 2022] GET /identity/v3/role_assignments?user.id=9a357d94a1a94d738b4a4f31c5dc680a&scope.system=all&role.id=6d053d567af44b6685295deb347f550e => generated 510 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:27.258237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47191141-cd91-47df-a1e1-c78556275743 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.258753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47191141-cd91-47df-a1e1-c78556275743 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.259172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59313-ea9b-418c-8dae-0096404a7627 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.259464 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59313-ea9b-418c-8dae-0096404a7627 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.259747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef59313-ea9b-418c-8dae-0096404a7627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.298255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-eef59313-ea9b-418c-8dae-0096404a7627 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:27.300064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-eef59313-ea9b-418c-8dae-0096404a7627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:27.342224 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47191141-cd91-47df-a1e1-c78556275743 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:27.344875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47191141-cd91-47df-a1e1-c78556275743 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:27.390408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 188/362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:27.391710 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d995c2b-08cb-477b-b7bf-cd0f01ea531b None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.392001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d995c2b-08cb-477b-b7bf-cd0f01ea531b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.392293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d995c2b-08cb-477b-b7bf-cd0f01ea531b None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.393717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 189/363] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:27.399667 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe9e40-a8f9-496f-a259-3b717c0c30c9 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.399996 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe9e40-a8f9-496f-a259-3b717c0c30c9 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.400231 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe9e40-a8f9-496f-a259-3b717c0c30c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.418271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-c3fe9e40-a8f9-496f-a259-3b717c0c30c9 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:27.419537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-c3fe9e40-a8f9-496f-a259-3b717c0c30c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:27.477502 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 175/364] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:27.478468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47191141-cd91-47df-a1e1-c78556275743 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:27.514213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 190/365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:27.515864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe9e40-a8f9-496f-a259-3b717c0c30c9 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:27.628596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47191141-cd91-47df-a1e1-c78556275743 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:27.644853 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe9e40-a8f9-496f-a259-3b717c0c30c9 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:27.669101 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478330-7776-4813-a247-bebc7d300913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:27.670097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478330-7776-4813-a247-bebc7d300913 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.670473 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478330-7776-4813-a247-bebc7d300913 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.670847 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478330-7776-4813-a247-bebc7d300913 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.688580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478330-7776-4813-a247-bebc7d300913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:27.691020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a6fc5-1289-4860-8a17-96831075993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:27.692310 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a6fc5-1289-4860-8a17-96831075993b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.693073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a6fc5-1289-4860-8a17-96831075993b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.693794 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a6fc5-1289-4860-8a17-96831075993b None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.696384 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a6fc5-1289-4860-8a17-96831075993b None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-41478330-7776-4813-a247-bebc7d300913 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:27.698289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:27.699526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:27.700519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:27.701500 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 176/366] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:27.702570 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478330-7776-4813-a247-bebc7d300913 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:27.702861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a6fc5-1289-4860-8a17-96831075993b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:27.705020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 191/367] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:27 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:27.706464 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a6fc5-1289-4860-8a17-96831075993b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:27.808745 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478330-7776-4813-a247-bebc7d300913 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:27.815895 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a6fc5-1289-4860-8a17-96831075993b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:27.844861 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc5531-e988-4a1a-b2f5-ef64cba20468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:27.846143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc5531-e988-4a1a-b2f5-ef64cba20468 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.846779 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc5531-e988-4a1a-b2f5-ef64cba20468 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.847818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc5531-e988-4a1a-b2f5-ef64cba20468 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.851593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:27.852271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:27.852553 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:27.852828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:27.853610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:27.853915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:27.855299 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:27.863689 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc5531-e988-4a1a-b2f5-ef64cba20468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:27.866665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 192/368] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:27.867509 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-e6dc5531-e988-4a1a-b2f5-ef64cba20468 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:27.872955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:27.874097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:27.874976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:27.877715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 177/369] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:28:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:27.879056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc5531-e988-4a1a-b2f5-ef64cba20468 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:27.988218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c629-3a59-48f0-881e-6bd878f69c1f None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:27.991657 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc5531-e988-4a1a-b2f5-ef64cba20468 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:28.022901 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b0962-e956-4565-8d66-ab5ee48bf628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:28.023850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b0962-e956-4565-8d66-ab5ee48bf628 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:28.024136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b0962-e956-4565-8d66-ab5ee48bf628 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:28.024415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b0962-e956-4565-8d66-ab5ee48bf628 None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:28.025694 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b0962-e956-4565-8d66-ab5ee48bf628 None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:28.027143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b0962-e956-4565-8d66-ab5ee48bf628 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:28.028789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 193/370] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:27 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:28.029203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f11a-2a4c-4ed2-9973-5883e5a5f25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:28.029590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b0962-e956-4565-8d66-ab5ee48bf628 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:28.030516 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f11a-2a4c-4ed2-9973-5883e5a5f25b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:28.031027 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f11a-2a4c-4ed2-9973-5883e5a5f25b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:28.031537 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f11a-2a4c-4ed2-9973-5883e5a5f25b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:28.109716 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960f11a-2a4c-4ed2-9973-5883e5a5f25b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:28.109716 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960f11a-2a4c-4ed2-9973-5883e5a5f25b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:28.133898 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b0962-e956-4565-8d66-ab5ee48bf628 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:28.168587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:28.169485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:28.169779 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:28.170054 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:28.170899 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:28.171209 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:28.172918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:28.181983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 194/371] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:28.182828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:28.214165 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 178/372] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:28.215125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f11a-2a4c-4ed2-9973-5883e5a5f25b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:28.285711 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735f92f-09e7-4cb3-9cd7-51214e869ac5 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:28.316562 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f11a-2a4c-4ed2-9973-5883e5a5f25b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:28.319141 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:28.319864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:28.320214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:28.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:28.321491 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] RBAC: Adding query filter params (group.id=5e27e058f298484bbc650a1b2c1ca9fc, role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=d6cb7aefad52496890bce00f2bb83781) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:28.321804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:28.323248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:28.328818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 195/373] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:28 2022] GET /identity/v3/role_assignments?group.id=5e27e058f298484bbc650a1b2c1ca9fc&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 260 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:28.329640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:28.353711 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdeef0-040b-422c-b724-27f13c65898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:28.354710 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdeef0-040b-422c-b724-27f13c65898f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:28.355004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdeef0-040b-422c-b724-27f13c65898f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:28.355286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdeef0-040b-422c-b724-27f13c65898f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:28.447036 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdeef0-040b-422c-b724-27f13c65898f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:28.447408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f83f6-ba7c-430a-b931-5b16adbcd937 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:28.448579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdeef0-040b-422c-b724-27f13c65898f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:28.492654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58c33-89e8-4439-9f20-1a621799fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:28.494045 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58c33-89e8-4439-9f20-1a621799fa65 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:28.494352 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58c33-89e8-4439-9f20-1a621799fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:28.494639 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58c33-89e8-4439-9f20-1a621799fa65 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:28.496272 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc58c33-89e8-4439-9f20-1a621799fa65 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:28.498426 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc58c33-89e8-4439-9f20-1a621799fa65 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:28.501266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 196/374] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:28 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:28.510629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58c33-89e8-4439-9f20-1a621799fa65 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:28.606671 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 179/375] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:28.613253 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdeef0-040b-422c-b724-27f13c65898f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:28.685975 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58c33-89e8-4439-9f20-1a621799fa65 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:28.726967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb258019-ed7a-4c26-ba7c-9fca4ed49276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:28.727839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258019-ed7a-4c26-ba7c-9fca4ed49276 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:28.728128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258019-ed7a-4c26-ba7c-9fca4ed49276 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:28.728409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258019-ed7a-4c26-ba7c-9fca4ed49276 None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:28.729800 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb258019-ed7a-4c26-ba7c-9fca4ed49276 None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:28.731409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb258019-ed7a-4c26-ba7c-9fca4ed49276 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:28.733895 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 197/376] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:28 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:28.740744 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb258019-ed7a-4c26-ba7c-9fca4ed49276 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:28.764049 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdeef0-040b-422c-b724-27f13c65898f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:28.841762 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14314d7-1e38-40d8-8724-09b07c62a72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:28.843069 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14314d7-1e38-40d8-8724-09b07c62a72d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:28.843577 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14314d7-1e38-40d8-8724-09b07c62a72d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:28.844086 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14314d7-1e38-40d8-8724-09b07c62a72d None admin] PATH_INFO: `/v3/projects/96210966ed2546dea7963761675d8627` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:28.845806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14314d7-1e38-40d8-8724-09b07c62a72d None admin] RBAC: Authorizing `identity:get_project(project_id=96210966ed2546dea7963761675d8627)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:28.847547 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14314d7-1e38-40d8-8724-09b07c62a72d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:28.852903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 180/377] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:28 2022] GET /identity/v3/projects/96210966ed2546dea7963761675d8627 => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:28.858838 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14314d7-1e38-40d8-8724-09b07c62a72d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:28.944726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb258019-ed7a-4c26-ba7c-9fca4ed49276 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:29.014370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:29.015482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.016548 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.017644 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] RBAC: Adding query filter params (group.id=5e27e058f298484bbc650a1b2c1ca9fc, role.id=02f09975329342eead41c01b150325c0, scope.project.id=d6cb7aefad52496890bce00f2bb83781) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:29.018185 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:29.020556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:29.030111 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 198/378] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:28 2022] GET /identity/v3/role_assignments?group.id=5e27e058f298484bbc650a1b2c1ca9fc&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=02f09975329342eead41c01b150325c0 => generated 260 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:29.047916 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14314d7-1e38-40d8-8724-09b07c62a72d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:29.105552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65745e5b-c0ec-4ecd-830a-62f27db60ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:29.106438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745e5b-c0ec-4ecd-830a-62f27db60ade None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.106721 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745e5b-c0ec-4ecd-830a-62f27db60ade None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.106994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745e5b-c0ec-4ecd-830a-62f27db60ade None admin] PATH_INFO: `/v3/groups/838bf9984b9f4333b43b15b7a1b97b9a` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.118635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65745e5b-c0ec-4ecd-830a-62f27db60ade None admin] RBAC: Authorizing `identity:get_group(group_id=838bf9984b9f4333b43b15b7a1b97b9a)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:29.120064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65745e5b-c0ec-4ecd-830a-62f27db60ade None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:29.122200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 181/379] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:28 2022] GET /identity/v3/groups/838bf9984b9f4333b43b15b7a1b97b9a => generated 229 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:29.128170 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:29.269820 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4ef28-720a-4713-bc35-cdad1b90abb3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:29.307825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7efa44-8c8c-4aae-9e68-96e5baaeb85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:29.308496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7efa44-8c8c-4aae-9e68-96e5baaeb85b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.308777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7efa44-8c8c-4aae-9e68-96e5baaeb85b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.309074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7efa44-8c8c-4aae-9e68-96e5baaeb85b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.310785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7efa44-8c8c-4aae-9e68-96e5baaeb85b None admin] RBAC: Adding query filter params (group.id=838bf9984b9f4333b43b15b7a1b97b9a, role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=96210966ed2546dea7963761675d8627) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:29.310785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7efa44-8c8c-4aae-9e68-96e5baaeb85b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:29.311691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7efa44-8c8c-4aae-9e68-96e5baaeb85b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:29.320477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 199/380] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:29 2022] GET /identity/v3/role_assignments?group.id=838bf9984b9f4333b43b15b7a1b97b9a&scope.project.id=96210966ed2546dea7963761675d8627&role.id=58c34600086c469b87bfa39b74dd2607 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:29.366856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192e30c-35ce-4d2b-a75d-c744f8675e39 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.367192 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192e30c-35ce-4d2b-a75d-c744f8675e39 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.367486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192e30c-35ce-4d2b-a75d-c744f8675e39 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.369350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 182/381] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:29.375070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17145-d85a-4c3b-b7b4-97be8caa8370 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.375360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17145-d85a-4c3b-b7b4-97be8caa8370 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.375642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17145-d85a-4c3b-b7b4-97be8caa8370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.395992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-baf17145-d85a-4c3b-b7b4-97be8caa8370 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:29.397376 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-baf17145-d85a-4c3b-b7b4-97be8caa8370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:29.424560 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c0e0-41d3-455b-8481-d5f9ff6c64a8 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.424913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c0e0-41d3-455b-8481-d5f9ff6c64a8 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.425145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c0e0-41d3-455b-8481-d5f9ff6c64a8 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.427402 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 183/382] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:29.437120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350956-ff8f-4550-be0b-2fcf0e5ebf2d None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.437120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350956-ff8f-4550-be0b-2fcf0e5ebf2d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.437120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44350956-ff8f-4550-be0b-2fcf0e5ebf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.466021 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-44350956-ff8f-4550-be0b-2fcf0e5ebf2d None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:29.467622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-44350956-ff8f-4550-be0b-2fcf0e5ebf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:29.525838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 200/383] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:29.558326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17145-d85a-4c3b-b7b4-97be8caa8370 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:29.576223 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 184/384] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:29.613998 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44350956-ff8f-4550-be0b-2fcf0e5ebf2d None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:29.711591 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17145-d85a-4c3b-b7b4-97be8caa8370 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:29.772666 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f5e7f-c7ee-41b6-9cda-ab5f64011102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:29.773497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f5e7f-c7ee-41b6-9cda-ab5f64011102 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.773785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f5e7f-c7ee-41b6-9cda-ab5f64011102 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.774063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f5e7f-c7ee-41b6-9cda-ab5f64011102 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.778017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44350956-ff8f-4550-be0b-2fcf0e5ebf2d None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:29.798237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f5e7f-c7ee-41b6-9cda-ab5f64011102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-cc9f5e7f-c7ee-41b6-9cda-ab5f64011102 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:29.808852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:29.809870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:29.810789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:29.819962 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 201/385] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:29.823570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f5e7f-c7ee-41b6-9cda-ab5f64011102 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:29.848341 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1de87-eac4-4535-a96e-a25f0f840910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:29.849571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1de87-eac4-4535-a96e-a25f0f840910 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:29.850067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1de87-eac4-4535-a96e-a25f0f840910 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:29.850580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1de87-eac4-4535-a96e-a25f0f840910 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:29.872762 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1de87-eac4-4535-a96e-a25f0f840910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-40c1de87-eac4-4535-a96e-a25f0f840910 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:29.893438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:29.895091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:29.895976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:29.899459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 185/386] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:29.907295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1de87-eac4-4535-a96e-a25f0f840910 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:30.021450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f5e7f-c7ee-41b6-9cda-ab5f64011102 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:30.071941 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1de87-eac4-4535-a96e-a25f0f840910 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:30.107665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:30.108347 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:30.108632 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:30.108908 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:30.109922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:30.110233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:30.110851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:30.111599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:30.111712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:30.111964 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:30.112314 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:30.113213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:30.113527 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:30.114904 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:30.124133 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 186/387] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:30.130101 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:30.131383 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 202/388] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:30.136665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:30.286671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f59c04-317e-40f9-9b63-00a771fee709 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:30.328887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55472f87-ba1c-419c-b025-0e0baff5753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:30.329792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55472f87-ba1c-419c-b025-0e0baff5753a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:30.330074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55472f87-ba1c-419c-b025-0e0baff5753a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:30.330349 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55472f87-ba1c-419c-b025-0e0baff5753a None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:30.332567 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55472f87-ba1c-419c-b025-0e0baff5753a None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:30.334042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55472f87-ba1c-419c-b025-0e0baff5753a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:30.335690 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 203/389] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:30 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:30.346066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55472f87-ba1c-419c-b025-0e0baff5753a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:30.355947 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339f4e7-6f23-426f-8432-0c3416242fc2 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:30.422659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767cb81-9831-4ed1-a680-7a6a2e1c2f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:30.423542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767cb81-9831-4ed1-a680-7a6a2e1c2f7e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:30.423831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767cb81-9831-4ed1-a680-7a6a2e1c2f7e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:30.424118 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767cb81-9831-4ed1-a680-7a6a2e1c2f7e None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:30.425901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767cb81-9831-4ed1-a680-7a6a2e1c2f7e None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:30.427366 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767cb81-9831-4ed1-a680-7a6a2e1c2f7e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:30.433253 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 187/390] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:30 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:30.434428 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767cb81-9831-4ed1-a680-7a6a2e1c2f7e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:30.449655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55472f87-ba1c-419c-b025-0e0baff5753a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:30.483113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d350-f207-478f-970a-1976f1e5538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:30.483977 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d350-f207-478f-970a-1976f1e5538d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:30.484254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d350-f207-478f-970a-1976f1e5538d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:30.484524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d350-f207-478f-970a-1976f1e5538d None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:30.485760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66d350-f207-478f-970a-1976f1e5538d None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:30.487203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66d350-f207-478f-970a-1976f1e5538d None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:30.489781 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 204/391] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:30 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:30.496421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d350-f207-478f-970a-1976f1e5538d None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:30.587802 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767cb81-9831-4ed1-a680-7a6a2e1c2f7e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:30.605998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d350-f207-478f-970a-1976f1e5538d None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:30.640739 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be89b21-4b08-4959-9006-a6e06d9d945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:30.641656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89b21-4b08-4959-9006-a6e06d9d945b None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:30.641993 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89b21-4b08-4959-9006-a6e06d9d945b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:30.642269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89b21-4b08-4959-9006-a6e06d9d945b None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530/groups/5e27e058f298484bbc650a1b2c1ca9fc/roles/251b2e26a5834411bf902ba50b5ec81f` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:30.644326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be89b21-4b08-4959-9006-a6e06d9d945b None admin] RBAC: Authorizing `identity:create_grant(project_id=3d209aef7bbd4477a103fd4064afe530, group_id=5e27e058f298484bbc650a1b2c1ca9fc, role_id=251b2e26a5834411bf902ba50b5ec81f)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:30.645742 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67b68d-3c5d-4527-9e0a-3b1956b9ef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:30.645973 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be89b21-4b08-4959-9006-a6e06d9d945b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:30.646618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67b68d-3c5d-4527-9e0a-3b1956b9ef51 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:30.646902 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67b68d-3c5d-4527-9e0a-3b1956b9ef51 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:30.647172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67b68d-3c5d-4527-9e0a-3b1956b9ef51 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:30.648415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67b68d-3c5d-4527-9e0a-3b1956b9ef51 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:30.649874 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67b68d-3c5d-4527-9e0a-3b1956b9ef51 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:30.651858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 188/392] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:30 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:30.657597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67b68d-3c5d-4527-9e0a-3b1956b9ef51 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:30.673667 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 205/393] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:28:30 2022] PUT /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530/groups/5e27e058f298484bbc650a1b2c1ca9fc/roles/251b2e26a5834411bf902ba50b5ec81f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:30.782619 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67b68d-3c5d-4527-9e0a-3b1956b9ef51 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:30.829875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df310f4-4260-4192-97e3-a6d6a07c6346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:30.830790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df310f4-4260-4192-97e3-a6d6a07c6346 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:30.831073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df310f4-4260-4192-97e3-a6d6a07c6346 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:30.831354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df310f4-4260-4192-97e3-a6d6a07c6346 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530/groups/5e27e058f298484bbc650a1b2c1ca9fc/roles/02f09975329342eead41c01b150325c0` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:30.836877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df310f4-4260-4192-97e3-a6d6a07c6346 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d209aef7bbd4477a103fd4064afe530, group_id=5e27e058f298484bbc650a1b2c1ca9fc, role_id=02f09975329342eead41c01b150325c0)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:30.838535 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df310f4-4260-4192-97e3-a6d6a07c6346 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:30.855744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 189/394] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:28:30 2022] PUT /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530/groups/5e27e058f298484bbc650a1b2c1ca9fc/roles/02f09975329342eead41c01b150325c0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:28:31.265293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9733fa0-2c57-4b8b-a3d2-5ef6cc4d025b None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:31.265830 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9733fa0-2c57-4b8b-a3d2-5ef6cc4d025b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:31.265915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9733fa0-2c57-4b8b-a3d2-5ef6cc4d025b None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:31.267530 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 206/395] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:31.273175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a613b-e9da-4331-9f03-540a46406b91 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:31.273474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a613b-e9da-4331-9f03-540a46406b91 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:31.273761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a613b-e9da-4331-9f03-540a46406b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:31.294460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-b73a613b-e9da-4331-9f03-540a46406b91 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:31.295763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-b73a613b-e9da-4331-9f03-540a46406b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:31.386209 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 190/396] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:31.421579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9733fa0-2c57-4b8b-a3d2-5ef6cc4d025b None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:31.547079 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9733fa0-2c57-4b8b-a3d2-5ef6cc4d025b None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:31.584429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443e42e-f12e-4383-b5a5-af2f6ce0af2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:31.585273 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e42e-f12e-4383-b5a5-af2f6ce0af2d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:31.585563 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e42e-f12e-4383-b5a5-af2f6ce0af2d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:31.585846 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e42e-f12e-4383-b5a5-af2f6ce0af2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:31.596515 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443e42e-f12e-4383-b5a5-af2f6ce0af2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-1443e42e-f12e-4383-b5a5-af2f6ce0af2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:31.605003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:31.606116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:31.607028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:31.609064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 207/397] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:31.614772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a613b-e9da-4331-9f03-540a46406b91 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:31.765208 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a613b-e9da-4331-9f03-540a46406b91 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:31.809694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:31.810494 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:31.810726 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:31.811061 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:31.811836 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] RBAC: Adding query filter params (name=member) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:31.812152 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:31.813574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:31.824145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 191/398] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:31.829575 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443e42e-f12e-4383-b5a5-af2f6ce0af2d None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:31.948300 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443e42e-f12e-4383-b5a5-af2f6ce0af2d None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:31.987060 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed9b8-7ce4-4293-80ba-1c97f2c4f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:31.987983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed9b8-7ce4-4293-80ba-1c97f2c4f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:31.988271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed9b8-7ce4-4293-80ba-1c97f2c4f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:31.988552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed9b8-7ce4-4293-80ba-1c97f2c4f9d2 None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:31.989871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ed9b8-7ce4-4293-80ba-1c97f2c4f9d2 None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:31.991361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ed9b8-7ce4-4293-80ba-1c97f2c4f9d2 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:31.993532 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 208/399] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:31 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:31.999280 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:32.119985 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff64fe7-4c58-45f8-8cb2-a3cd772c532a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:32.162827 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc729998-6727-4aa2-a406-3f9bf86eb5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:32.163716 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc729998-6727-4aa2-a406-3f9bf86eb5c8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.164004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc729998-6727-4aa2-a406-3f9bf86eb5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.164287 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc729998-6727-4aa2-a406-3f9bf86eb5c8 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.165543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc729998-6727-4aa2-a406-3f9bf86eb5c8 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:32.167001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc729998-6727-4aa2-a406-3f9bf86eb5c8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:32.169096 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 192/400] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:31 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:32.174412 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed9b8-7ce4-4293-80ba-1c97f2c4f9d2 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:32.186147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b45e-86c4-4590-95d0-16eccb77c2f2 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.186495 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b45e-86c4-4590-95d0-16eccb77c2f2 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.186713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b45e-86c4-4590-95d0-16eccb77c2f2 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.190423 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 193/401] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:32.196298 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c8e1-238b-4671-b3b1-4c43f4bc16b7 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.196658 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c8e1-238b-4671-b3b1-4c43f4bc16b7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.196864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c8e1-238b-4671-b3b1-4c43f4bc16b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.219846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-a0c4c8e1-238b-4671-b3b1-4c43f4bc16b7 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:32.221426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-a0c4c8e1-238b-4671-b3b1-4c43f4bc16b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:32.346727 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed9b8-7ce4-4293-80ba-1c97f2c4f9d2 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:32.348748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 194/402] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:32.381692 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99347d36-1a74-4da9-8ffa-838eafe0dff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:32.382586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347d36-1a74-4da9-8ffa-838eafe0dff1 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.382874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347d36-1a74-4da9-8ffa-838eafe0dff1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.383151 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99347d36-1a74-4da9-8ffa-838eafe0dff1 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781/groups/5e27e058f298484bbc650a1b2c1ca9fc/roles/251b2e26a5834411bf902ba50b5ec81f` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.385243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99347d36-1a74-4da9-8ffa-838eafe0dff1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cb7aefad52496890bce00f2bb83781, group_id=5e27e058f298484bbc650a1b2c1ca9fc, role_id=251b2e26a5834411bf902ba50b5ec81f)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:32.386844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99347d36-1a74-4da9-8ffa-838eafe0dff1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:32.389550 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4c8e1-238b-4671-b3b1-4c43f4bc16b7 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:32.406176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 209/403] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:28:32 2022] PUT /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781/groups/5e27e058f298484bbc650a1b2c1ca9fc/roles/251b2e26a5834411bf902ba50b5ec81f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:32.481093 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ad440-9bec-42b5-ad29-9498ff7a7155 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.481744 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ad440-9bec-42b5-ad29-9498ff7a7155 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.482522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ad440-9bec-42b5-ad29-9498ff7a7155 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.484823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 210/404] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:32.493363 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594779a7-42bb-4fd8-b149-108c5800474b None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.494375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594779a7-42bb-4fd8-b149-108c5800474b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.495254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594779a7-42bb-4fd8-b149-108c5800474b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.516718 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-594779a7-42bb-4fd8-b149-108c5800474b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:32.518325 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-594779a7-42bb-4fd8-b149-108c5800474b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:32.541733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4c8e1-238b-4671-b3b1-4c43f4bc16b7 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:32.578518 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e45a5d-2e7b-4f1b-8193-b789450705ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:32.579367 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e45a5d-2e7b-4f1b-8193-b789450705ed None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.579650 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e45a5d-2e7b-4f1b-8193-b789450705ed None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.580334 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e45a5d-2e7b-4f1b-8193-b789450705ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.591177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e45a5d-2e7b-4f1b-8193-b789450705ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:32.597013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 211/405] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-84e45a5d-2e7b-4f1b-8193-b789450705ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:32.605179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:32.606246 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:32.607405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 195/406] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:28:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:32.610731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594779a7-42bb-4fd8-b149-108c5800474b None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:32.626456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e45a5d-2e7b-4f1b-8193-b789450705ed None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:32.720676 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594779a7-42bb-4fd8-b149-108c5800474b None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:32.732633 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e45a5d-2e7b-4f1b-8193-b789450705ed None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:32.766817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:32.767892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.768492 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.769088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.770105 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:32.770632 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:32.772246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:32.782010 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f5b0-b904-4a87-9cf7-94d807770d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:32.782886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f5b0-b904-4a87-9cf7-94d807770d22 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.783165 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f5b0-b904-4a87-9cf7-94d807770d22 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.783484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f5b0-b904-4a87-9cf7-94d807770d22 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.793409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 212/407] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:32.794509 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0f5b0-b904-4a87-9cf7-94d807770d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:32.794942 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-bec0f5b0-b904-4a87-9cf7-94d807770d22 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:32.803335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:32.804356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:32.805265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 196/408] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:32.810503 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f5b0-b904-4a87-9cf7-94d807770d22 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:32.914121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f5b0-b904-4a87-9cf7-94d807770d22 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:32.935246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75911bb4-7e30-4fe8-bdc7-eeb4af0dbd81 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:32.950986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:32.950986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.950986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.950986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.951986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:32.952395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:32.954488 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:32.964978 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 197/409] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:32.970571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:32.980064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8898b0-2dcc-434a-8e16-db3892c3bb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:32.981843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8898b0-2dcc-434a-8e16-db3892c3bb5f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:32.982612 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8898b0-2dcc-434a-8e16-db3892c3bb5f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:32.983138 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8898b0-2dcc-434a-8e16-db3892c3bb5f None admin] PATH_INFO: `/v3/groups/838bf9984b9f4333b43b15b7a1b97b9a` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:32.984591 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8898b0-2dcc-434a-8e16-db3892c3bb5f None admin] RBAC: Authorizing `identity:get_group(group_id=838bf9984b9f4333b43b15b7a1b97b9a)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:32.986413 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8898b0-2dcc-434a-8e16-db3892c3bb5f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:32.988882 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 213/410] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:32 2022] GET /identity/v3/groups/838bf9984b9f4333b43b15b7a1b97b9a => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:32.999246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8898b0-2dcc-434a-8e16-db3892c3bb5f None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:33.089430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9f3fe-e80f-4c12-8ae1-dc545451c5ca None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:33.137841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45164c2d-68a8-4d38-a2f6-d8332bcb3431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:33.138725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45164c2d-68a8-4d38-a2f6-d8332bcb3431 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:33.139008 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45164c2d-68a8-4d38-a2f6-d8332bcb3431 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:33.139282 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45164c2d-68a8-4d38-a2f6-d8332bcb3431 None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:33.140575 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45164c2d-68a8-4d38-a2f6-d8332bcb3431 None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:33.142058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45164c2d-68a8-4d38-a2f6-d8332bcb3431 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:33.144029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 198/411] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:32 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:33.149426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45164c2d-68a8-4d38-a2f6-d8332bcb3431 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:33.163928 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8898b0-2dcc-434a-8e16-db3892c3bb5f None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:33.203576 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118c75a-9775-4992-b54a-15b4cdc7e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:33.204576 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c75a-9775-4992-b54a-15b4cdc7e139 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:33.204978 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c75a-9775-4992-b54a-15b4cdc7e139 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:33.205402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c75a-9775-4992-b54a-15b4cdc7e139 None admin] PATH_INFO: `/v3/projects/96210966ed2546dea7963761675d8627` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:33.207336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118c75a-9775-4992-b54a-15b4cdc7e139 None admin] RBAC: Authorizing `identity:get_project(project_id=96210966ed2546dea7963761675d8627)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:33.208904 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118c75a-9775-4992-b54a-15b4cdc7e139 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:33.211095 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 214/412] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:32 2022] GET /identity/v3/projects/96210966ed2546dea7963761675d8627 => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:33.227497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118c75a-9775-4992-b54a-15b4cdc7e139 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:33.276410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45164c2d-68a8-4d38-a2f6-d8332bcb3431 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:33.316748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670b630-a720-4666-b86b-945f0946ec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:33.317640 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670b630-a720-4666-b86b-945f0946ec23 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:33.317921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670b630-a720-4666-b86b-945f0946ec23 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:33.318200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670b630-a720-4666-b86b-945f0946ec23 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:33.320459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670b630-a720-4666-b86b-945f0946ec23 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:33.321943 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670b630-a720-4666-b86b-945f0946ec23 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:33.329394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 199/413] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:33 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:33.332639 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670b630-a720-4666-b86b-945f0946ec23 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:33.390704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118c75a-9775-4992-b54a-15b4cdc7e139 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:33.455544 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c090e-1ebd-4b14-a82a-6a76fa537314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:33.456351 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c090e-1ebd-4b14-a82a-6a76fa537314 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:33.457231 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c090e-1ebd-4b14-a82a-6a76fa537314 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:33.457508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c090e-1ebd-4b14-a82a-6a76fa537314 None admin] PATH_INFO: `/v3/projects/96210966ed2546dea7963761675d8627/groups/838bf9984b9f4333b43b15b7a1b97b9a/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:33.464601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c090e-1ebd-4b14-a82a-6a76fa537314 None admin] RBAC: Authorizing `identity:create_grant(project_id=96210966ed2546dea7963761675d8627, group_id=838bf9984b9f4333b43b15b7a1b97b9a, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:33.467253 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c090e-1ebd-4b14-a82a-6a76fa537314 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:33.491880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 215/414] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:28:33 2022] PUT /identity/v3/projects/96210966ed2546dea7963761675d8627/groups/838bf9984b9f4333b43b15b7a1b97b9a/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:33.530829 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670b630-a720-4666-b86b-945f0946ec23 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:33.582147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1251d7e-2675-44fd-934d-1ce32c3577ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:33.583103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1251d7e-2675-44fd-934d-1ce32c3577ed None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:33.583395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1251d7e-2675-44fd-934d-1ce32c3577ed None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:33.583739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1251d7e-2675-44fd-934d-1ce32c3577ed None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781/groups/5e27e058f298484bbc650a1b2c1ca9fc/roles/02f09975329342eead41c01b150325c0` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:33.587855 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1251d7e-2675-44fd-934d-1ce32c3577ed None admin] RBAC: Authorizing `identity:create_grant(project_id=d6cb7aefad52496890bce00f2bb83781, group_id=5e27e058f298484bbc650a1b2c1ca9fc, role_id=02f09975329342eead41c01b150325c0)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:33.589406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1251d7e-2675-44fd-934d-1ce32c3577ed None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:33.618580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 200/415] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:28:33 2022] PUT /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781/groups/5e27e058f298484bbc650a1b2c1ca9fc/roles/02f09975329342eead41c01b150325c0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:33.727109 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c56aeb-edab-4fc5-b077-d11abbb0e705 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:33.727454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c56aeb-edab-4fc5-b077-d11abbb0e705 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:33.727645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c56aeb-edab-4fc5-b077-d11abbb0e705 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:33.729454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 216/416] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:33.735552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7d24-6f4b-4952-a51b-b509919e25dc None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:33.735871 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7d24-6f4b-4952-a51b-b509919e25dc None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:33.736125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7d24-6f4b-4952-a51b-b509919e25dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:33.755344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-26fe7d24-6f4b-4952-a51b-b509919e25dc None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:33.756631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-26fe7d24-6f4b-4952-a51b-b509919e25dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:33.848681 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 201/417] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:33.873132 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c56aeb-edab-4fc5-b077-d11abbb0e705 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:33.986310 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c56aeb-edab-4fc5-b077-d11abbb0e705 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:34.021351 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae88ae-2e02-40b4-a4a5-78f091e07e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:34.022163 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae88ae-2e02-40b4-a4a5-78f091e07e19 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:34.022437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae88ae-2e02-40b4-a4a5-78f091e07e19 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:34.022714 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae88ae-2e02-40b4-a4a5-78f091e07e19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:34.030139 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862db69c-9062-4b06-a835-9bce585490f3 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:34.030430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862db69c-9062-4b06-a835-9bce585490f3 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:34.030703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862db69c-9062-4b06-a835-9bce585490f3 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:34.033664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 202/418] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:34.038067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75f7c4-18bb-466b-9b04-af357614a65b None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:34.038363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75f7c4-18bb-466b-9b04-af357614a65b None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:34.038653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75f7c4-18bb-466b-9b04-af357614a65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:34.039213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae88ae-2e02-40b4-a4a5-78f091e07e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-99ae88ae-2e02-40b4-a4a5-78f091e07e19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:34.047394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:34.048482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:34.049392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:34.053832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 217/419] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:34.059314 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-9b75f7c4-18bb-466b-9b04-af357614a65b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:34.059579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae88ae-2e02-40b4-a4a5-78f091e07e19 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:34.060534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-9b75f7c4-18bb-466b-9b04-af357614a65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:34.173192 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 203/420] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:34.204561 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae88ae-2e02-40b4-a4a5-78f091e07e19 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:34.245279 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75f7c4-18bb-466b-9b04-af357614a65b None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:34.254501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:34.255481 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:34.256006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:34.256501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:34.257525 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:34.258062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:34.259673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:34.272251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 218/421] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:34.288871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:34.380295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75f7c4-18bb-466b-9b04-af357614a65b None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:34.436265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306b5c3-343e-4ef7-b38f-b3372284a841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:34.437119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306b5c3-343e-4ef7-b38f-b3372284a841 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:34.437409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306b5c3-343e-4ef7-b38f-b3372284a841 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:34.437688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306b5c3-343e-4ef7-b38f-b3372284a841 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:34.448378 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306b5c3-343e-4ef7-b38f-b3372284a841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-3306b5c3-343e-4ef7-b38f-b3372284a841 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:34.456600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:34.457702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:34.458618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 204/422] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:34.493407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306b5c3-343e-4ef7-b38f-b3372284a841 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:34.503596 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9800a-cac5-4e1e-9f7e-b3e16a04d0e4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:34.544713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76754-673b-48db-af1a-a960b3564166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:34.545784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76754-673b-48db-af1a-a960b3564166 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:34.546311 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76754-673b-48db-af1a-a960b3564166 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:34.546805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76754-673b-48db-af1a-a960b3564166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:34.604081 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306b5c3-343e-4ef7-b38f-b3372284a841 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:34.638677 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:34.639359 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:34.639648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:34.639930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:34.640695 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:34.641014 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:34.642453 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:34.651556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 205/423] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:34.657143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:34.671140 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e76754-673b-48db-af1a-a960b3564166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:34.672640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e76754-673b-48db-af1a-a960b3564166 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:34.821222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a10ec-b687-4316-89d4-461b0e12b8c6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:34.872162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 219/424] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:28:34.878014 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76754-673b-48db-af1a-a960b3564166 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:34.884826 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17dc85-39c0-436f-9540-984838d52fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:34.889592 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17dc85-39c0-436f-9540-984838d52fbf None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:34.890147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17dc85-39c0-436f-9540-984838d52fbf None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:34.890668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17dc85-39c0-436f-9540-984838d52fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:34.997717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76754-673b-48db-af1a-a960b3564166 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:35.043199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18270858-894a-4584-baab-e21d07273272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:35.044518 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18270858-894a-4584-baab-e21d07273272 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:35.045072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18270858-894a-4584-baab-e21d07273272 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:35.045573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18270858-894a-4584-baab-e21d07273272 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:35.047231 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18270858-894a-4584-baab-e21d07273272 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:35.048990 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18270858-894a-4584-baab-e21d07273272 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:35.052439 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 220/425] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:34 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:35.060491 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18270858-894a-4584-baab-e21d07273272 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:35.078995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17dc85-39c0-436f-9540-984838d52fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:35.080395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17dc85-39c0-436f-9540-984838d52fbf None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:35.232868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 206/426] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:35.238757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17dc85-39c0-436f-9540-984838d52fbf None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:35.251934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18270858-894a-4584-baab-e21d07273272 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:35.291463 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0c779-4cc6-455f-a22f-1ff5ce5d2557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:35.292686 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0c779-4cc6-455f-a22f-1ff5ce5d2557 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:35.293243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0c779-4cc6-455f-a22f-1ff5ce5d2557 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:35.293783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0c779-4cc6-455f-a22f-1ff5ce5d2557 None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:35.295675 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0c779-4cc6-455f-a22f-1ff5ce5d2557 None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:35.297385 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0c779-4cc6-455f-a22f-1ff5ce5d2557 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:35.299574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 221/427] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:35 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:35.306308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0c779-4cc6-455f-a22f-1ff5ce5d2557 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:35.415136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0c779-4cc6-455f-a22f-1ff5ce5d2557 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:35.455963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:35.457617 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:35.458153 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:35.458668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:35.459789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] RBAC: Adding query filter params (group.id=5e27e058f298484bbc650a1b2c1ca9fc, role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=3d209aef7bbd4477a103fd4064afe530) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:35.460436 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:35.462193 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:35.471851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 222/428] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:35 2022] GET /identity/v3/role_assignments?group.id=5e27e058f298484bbc650a1b2c1ca9fc&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:35.505093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17dc85-39c0-436f-9540-984838d52fbf None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:35.566482 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5206f527-fecb-458f-9508-99c78264c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:35.567379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206f527-fecb-458f-9508-99c78264c675 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:35.567660 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206f527-fecb-458f-9508-99c78264c675 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:35.567935 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5206f527-fecb-458f-9508-99c78264c675 None admin] PATH_INFO: `/v3/projects/3d209aef7bbd4477a103fd4064afe530` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:35.569309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206f527-fecb-458f-9508-99c78264c675 None admin] RBAC: Authorizing `identity:get_project(project_id=3d209aef7bbd4477a103fd4064afe530)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:35.570766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5206f527-fecb-458f-9508-99c78264c675 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:35.572870 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 207/429] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:35 2022] GET /identity/v3/projects/3d209aef7bbd4477a103fd4064afe530 => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:35.578276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:35.640110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58dd747-1774-4431-b6e5-5278ee2bb891 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:35.640478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58dd747-1774-4431-b6e5-5278ee2bb891 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:35.640670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58dd747-1774-4431-b6e5-5278ee2bb891 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:35.642581 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 208/430] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:35.653963 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71079a6b-c8cd-4a9c-b582-a238c5b6f6a9 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:35.654322 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71079a6b-c8cd-4a9c-b582-a238c5b6f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:35.654541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71079a6b-c8cd-4a9c-b582-a238c5b6f6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:35.690314 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-71079a6b-c8cd-4a9c-b582-a238c5b6f6a9 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:35.692045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-71079a6b-c8cd-4a9c-b582-a238c5b6f6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:35.718260 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9537568-29a2-462b-9161-8a3d5fb4c084 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:35.762285 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143698b9-402e-4b74-b9f9-80ccbe9c9618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:35.763748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143698b9-402e-4b74-b9f9-80ccbe9c9618 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:35.764508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143698b9-402e-4b74-b9f9-80ccbe9c9618 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:35.764956 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143698b9-402e-4b74-b9f9-80ccbe9c9618 None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:35.766516 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143698b9-402e-4b74-b9f9-80ccbe9c9618 None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:35.769536 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143698b9-402e-4b74-b9f9-80ccbe9c9618 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:35.772225 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 223/431] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:35 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:35.778884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143698b9-402e-4b74-b9f9-80ccbe9c9618 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:35.797915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 209/432] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:35.830587 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71079a6b-c8cd-4a9c-b582-a238c5b6f6a9 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:35.902845 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143698b9-402e-4b74-b9f9-80ccbe9c9618 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:35.955194 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:35.955787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:35.956073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:35.956356 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:35.957275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] RBAC: Adding query filter params (group.id=5e27e058f298484bbc650a1b2c1ca9fc, role.id=02f09975329342eead41c01b150325c0, scope.project.id=3d209aef7bbd4477a103fd4064afe530) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:35.957594 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:35.959042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:35.965357 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71079a6b-c8cd-4a9c-b582-a238c5b6f6a9 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:35.965862 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 224/433] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:35 2022] GET /identity/v3/role_assignments?group.id=5e27e058f298484bbc650a1b2c1ca9fc&scope.project.id=3d209aef7bbd4477a103fd4064afe530&role.id=02f09975329342eead41c01b150325c0 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:36.010154 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793c810-1946-4760-a41a-7e250bc63ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:36.010989 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793c810-1946-4760-a41a-7e250bc63ca5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.011279 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793c810-1946-4760-a41a-7e250bc63ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.011561 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793c810-1946-4760-a41a-7e250bc63ca5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.024026 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793c810-1946-4760-a41a-7e250bc63ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-9793c810-1946-4760-a41a-7e250bc63ca5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:36.033309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:36.034360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:36.037961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 210/434] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:36.041418 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:36.215374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43ce38-fb11-495c-9a98-d685b305ee95 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:36.255747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae0350-7872-450a-89f5-c0e81c0310db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:36.256445 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0350-7872-450a-89f5-c0e81c0310db None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.256732 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0350-7872-450a-89f5-c0e81c0310db None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.257013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae0350-7872-450a-89f5-c0e81c0310db None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.257858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae0350-7872-450a-89f5-c0e81c0310db None admin] RBAC: Adding query filter params (name=member) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:36.258172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae0350-7872-450a-89f5-c0e81c0310db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:36.259650 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae0350-7872-450a-89f5-c0e81c0310db None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:36.271026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 225/435] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:28:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:36.277369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793c810-1946-4760-a41a-7e250bc63ca5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:36.368242 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523438-4e82-46ae-962a-c03fba979c10 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.368518 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523438-4e82-46ae-962a-c03fba979c10 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.368794 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523438-4e82-46ae-962a-c03fba979c10 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.370574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 226/436] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:36.375877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5361b6-f8ad-402c-9bde-eacd8693fb91 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.376479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5361b6-f8ad-402c-9bde-eacd8693fb91 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.376983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5361b6-f8ad-402c-9bde-eacd8693fb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.390455 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793c810-1946-4760-a41a-7e250bc63ca5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:36.396620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-ed5361b6-f8ad-402c-9bde-eacd8693fb91 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:36.397855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-ed5361b6-f8ad-402c-9bde-eacd8693fb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:36.428936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d2ee4-c0d4-4e30-b5a5-37bed2faaf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:36.429647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d2ee4-c0d4-4e30-b5a5-37bed2faaf7b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.429954 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d2ee4-c0d4-4e30-b5a5-37bed2faaf7b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.430232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d2ee4-c0d4-4e30-b5a5-37bed2faaf7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.490512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 227/437] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:36.493673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926067-e087-464d-a861-6f953e2ee94d None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.494206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926067-e087-464d-a861-6f953e2ee94d None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.494734 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926067-e087-464d-a861-6f953e2ee94d None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.496601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 228/438] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:36.503185 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfb906-029d-43a4-9bd5-5d871050b25b None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.503532 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfb906-029d-43a4-9bd5-5d871050b25b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.503823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfb906-029d-43a4-9bd5-5d871050b25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.523181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-58cfb906-029d-43a4-9bd5-5d871050b25b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:36.524613 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-58cfb906-029d-43a4-9bd5-5d871050b25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:36.530833 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d2ee4-c0d4-4e30-b5a5-37bed2faaf7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:36.532297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d2ee4-c0d4-4e30-b5a5-37bed2faaf7b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:36.596259 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 229/439] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:36.598213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfb906-029d-43a4-9bd5-5d871050b25b None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:36.645891 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 211/440] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:36.647072 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d2ee4-c0d4-4e30-b5a5-37bed2faaf7b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:36.698390 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfb906-029d-43a4-9bd5-5d871050b25b None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:36.731273 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b6e4-a138-469c-9d75-6ecc80a02dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:36.732072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b6e4-a138-469c-9d75-6ecc80a02dd4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.732621 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b6e4-a138-469c-9d75-6ecc80a02dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.732898 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b6e4-a138-469c-9d75-6ecc80a02dd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.742963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596b6e4-a138-469c-9d75-6ecc80a02dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:36.747085 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d2ee4-c0d4-4e30-b5a5-37bed2faaf7b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-e596b6e4-a138-469c-9d75-6ecc80a02dd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:36.750554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:36.751498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:36.752497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 230/441] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:28:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:36.753796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b6e4-a138-469c-9d75-6ecc80a02dd4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:36.781061 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8d870-80df-4787-9b2e-89fbd5da72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:36.781885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8d870-80df-4787-9b2e-89fbd5da72d0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.782175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8d870-80df-4787-9b2e-89fbd5da72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.782458 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8d870-80df-4787-9b2e-89fbd5da72d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.792383 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8d870-80df-4787-9b2e-89fbd5da72d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-f5c8d870-80df-4787-9b2e-89fbd5da72d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:36.800579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:36.801688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:36.802574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 212/442] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:36.803294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8d870-80df-4787-9b2e-89fbd5da72d0 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:36.852494 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b6e4-a138-469c-9d75-6ecc80a02dd4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:36.885265 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a4255-af97-40c3-b08d-137cf966c387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:36.886132 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4255-af97-40c3-b08d-137cf966c387 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.886420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4255-af97-40c3-b08d-137cf966c387 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.886700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4255-af97-40c3-b08d-137cf966c387 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.887927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a4255-af97-40c3-b08d-137cf966c387 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:36.889418 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a4255-af97-40c3-b08d-137cf966c387 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:36.896284 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 231/443] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:36 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:36.896284 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a4255-af97-40c3-b08d-137cf966c387 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:36.899883 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8d870-80df-4787-9b2e-89fbd5da72d0 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:36.932210 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:36.932887 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:36.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:36.933490 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:36.934252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:36.934567 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:36.935963 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:36.944415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 213/444] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:36.945257 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:36.989791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a4255-af97-40c3-b08d-137cf966c387 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.022701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.023292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.023575 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.023851 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.024610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:37.024924 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.026342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.034367 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 232/445] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:37.034973 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.047477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3295-2021-4615-9c9e-7cd7a339543d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.081597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ec5f1-c221-4e4e-ba6a-b88f2cf8a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.082460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ec5f1-c221-4e4e-ba6a-b88f2cf8a6b3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.082747 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ec5f1-c221-4e4e-ba6a-b88f2cf8a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.083029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ec5f1-c221-4e4e-ba6a-b88f2cf8a6b3 None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.084249 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ec5f1-c221-4e4e-ba6a-b88f2cf8a6b3 None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.085754 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ec5f1-c221-4e4e-ba6a-b88f2cf8a6b3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.092429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 214/446] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:36 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:37.092429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ec5f1-c221-4e4e-ba6a-b88f2cf8a6b3 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.141394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53257e3b-8ba2-468b-a740-ea2a3986d7f1 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.176289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7b65f-985d-4e97-8bf6-dc611e9bad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.176936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7b65f-985d-4e97-8bf6-dc611e9bad03 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.177288 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7b65f-985d-4e97-8bf6-dc611e9bad03 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.177569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7b65f-985d-4e97-8bf6-dc611e9bad03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.190140 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ec5f1-c221-4e4e-ba6a-b88f2cf8a6b3 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.222870 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8afc38-0bcd-4313-b9ce-ce36c015acba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.223525 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8afc38-0bcd-4313-b9ce-ce36c015acba None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.223811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8afc38-0bcd-4313-b9ce-ce36c015acba None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.224088 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8afc38-0bcd-4313-b9ce-ce36c015acba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.246433 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7b65f-985d-4e97-8bf6-dc611e9bad03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.247890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7b65f-985d-4e97-8bf6-dc611e9bad03 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.289181 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8afc38-0bcd-4313-b9ce-ce36c015acba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.290897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8afc38-0bcd-4313-b9ce-ce36c015acba None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.343820 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 233/447] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:37.344660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7b65f-985d-4e97-8bf6-dc611e9bad03 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.384443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 215/448] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:37.385346 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8afc38-0bcd-4313-b9ce-ce36c015acba None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.442737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7b65f-985d-4e97-8bf6-dc611e9bad03 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.474595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.475238 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.475520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.475792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.476637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] RBAC: Adding query filter params (group.id=5e27e058f298484bbc650a1b2c1ca9fc, role.id=251b2e26a5834411bf902ba50b5ec81f, scope.project.id=d6cb7aefad52496890bce00f2bb83781) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:37.476950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.478431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.484285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8afc38-0bcd-4313-b9ce-ce36c015acba None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.497314 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 234/449] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/role_assignments?group.id=5e27e058f298484bbc650a1b2c1ca9fc&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=251b2e26a5834411bf902ba50b5ec81f => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:37.497314 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.522296 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa84b3-950b-4788-b671-9e8fb62e5c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.523757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa84b3-950b-4788-b671-9e8fb62e5c4f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.524106 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa84b3-950b-4788-b671-9e8fb62e5c4f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.524398 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa84b3-950b-4788-b671-9e8fb62e5c4f None admin] PATH_INFO: `/v3/projects/96210966ed2546dea7963761675d8627` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.525760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa84b3-950b-4788-b671-9e8fb62e5c4f None admin] RBAC: Authorizing `identity:get_project(project_id=96210966ed2546dea7963761675d8627)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.527373 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa84b3-950b-4788-b671-9e8fb62e5c4f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.529382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 216/450] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/projects/96210966ed2546dea7963761675d8627 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:37.535365 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa84b3-950b-4788-b671-9e8fb62e5c4f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.606539 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb3a66-4882-4c92-bec2-9bb1b93b6ba0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.640912 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa84b3-950b-4788-b671-9e8fb62e5c4f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.642348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8de27-80f6-49f2-a497-4e1244c1fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.643229 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8de27-80f6-49f2-a497-4e1244c1fb02 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.643512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8de27-80f6-49f2-a497-4e1244c1fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.643787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8de27-80f6-49f2-a497-4e1244c1fb02 None admin] PATH_INFO: `/v3/projects/d6cb7aefad52496890bce00f2bb83781` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.645217 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8de27-80f6-49f2-a497-4e1244c1fb02 None admin] RBAC: Authorizing `identity:get_project(project_id=d6cb7aefad52496890bce00f2bb83781)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.646669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8de27-80f6-49f2-a497-4e1244c1fb02 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.648699 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 235/451] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/projects/d6cb7aefad52496890bce00f2bb83781 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:37.654210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8de27-80f6-49f2-a497-4e1244c1fb02 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.684023 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0673be-bd04-4803-83bb-1c6f4ec388d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.685015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0673be-bd04-4803-83bb-1c6f4ec388d4 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.685343 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0673be-bd04-4803-83bb-1c6f4ec388d4 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.685657 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0673be-bd04-4803-83bb-1c6f4ec388d4 None admin] PATH_INFO: `/v3/groups/838bf9984b9f4333b43b15b7a1b97b9a` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.687844 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0673be-bd04-4803-83bb-1c6f4ec388d4 None admin] RBAC: Authorizing `identity:get_group(group_id=838bf9984b9f4333b43b15b7a1b97b9a)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.690432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0673be-bd04-4803-83bb-1c6f4ec388d4 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.692484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 217/452] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/groups/838bf9984b9f4333b43b15b7a1b97b9a => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:37.700391 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0673be-bd04-4803-83bb-1c6f4ec388d4 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.767554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8de27-80f6-49f2-a497-4e1244c1fb02 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.803769 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215a7ac-d03a-4892-9cdf-7bdd1f99a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.804631 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215a7ac-d03a-4892-9cdf-7bdd1f99a77b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.804911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215a7ac-d03a-4892-9cdf-7bdd1f99a77b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.805106 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0673be-bd04-4803-83bb-1c6f4ec388d4 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.805238 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215a7ac-d03a-4892-9cdf-7bdd1f99a77b None admin] PATH_INFO: `/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.806551 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215a7ac-d03a-4892-9cdf-7bdd1f99a77b None admin] RBAC: Authorizing `identity:get_group(group_id=5e27e058f298484bbc650a1b2c1ca9fc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.807927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215a7ac-d03a-4892-9cdf-7bdd1f99a77b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.809935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 236/453] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/groups/5e27e058f298484bbc650a1b2c1ca9fc => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:37.815405 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215a7ac-d03a-4892-9cdf-7bdd1f99a77b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:37.840836 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd09df-650a-4513-88fb-a3d71ef953c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.842228 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd09df-650a-4513-88fb-a3d71ef953c1 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.842228 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd09df-650a-4513-88fb-a3d71ef953c1 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.842228 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd09df-650a-4513-88fb-a3d71ef953c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.842897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd09df-650a-4513-88fb-a3d71ef953c1 None admin] RBAC: Adding query filter params (group.id=838bf9984b9f4333b43b15b7a1b97b9a, role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=96210966ed2546dea7963761675d8627) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:37.843209 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd09df-650a-4513-88fb-a3d71ef953c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.844658 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd09df-650a-4513-88fb-a3d71ef953c1 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.852065 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 218/454] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/role_assignments?group.id=838bf9984b9f4333b43b15b7a1b97b9a&scope.project.id=96210966ed2546dea7963761675d8627&role.id=58c34600086c469b87bfa39b74dd2607 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:37.924554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215a7ac-d03a-4892-9cdf-7bdd1f99a77b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:37.959620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b9fdf-0779-4c67-9a84-4a86ced07bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:37.960284 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b9fdf-0779-4c67-9a84-4a86ced07bde None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:37.960565 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b9fdf-0779-4c67-9a84-4a86ced07bde None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:37.960840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b9fdf-0779-4c67-9a84-4a86ced07bde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:37.961728 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b9fdf-0779-4c67-9a84-4a86ced07bde None admin] RBAC: Adding query filter params (group.id=5e27e058f298484bbc650a1b2c1ca9fc, role.id=02f09975329342eead41c01b150325c0, scope.project.id=d6cb7aefad52496890bce00f2bb83781) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:37.962040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b9fdf-0779-4c67-9a84-4a86ced07bde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:37.963702 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b9fdf-0779-4c67-9a84-4a86ced07bde None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:37.970088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 237/455] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:28:37 2022] GET /identity/v3/role_assignments?group.id=5e27e058f298484bbc650a1b2c1ca9fc&scope.project.id=d6cb7aefad52496890bce00f2bb83781&role.id=02f09975329342eead41c01b150325c0 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:40.680179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e179-086e-4f2c-83cb-52c757be5ed2 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:40.680712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e179-086e-4f2c-83cb-52c757be5ed2 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:40.680712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e179-086e-4f2c-83cb-52c757be5ed2 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:40.682359 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 219/456] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:40.687835 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdda5-b71c-4508-ae87-fcf74aa2cfe8 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:40.688749 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdda5-b71c-4508-ae87-fcf74aa2cfe8 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:40.689271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdda5-b71c-4508-ae87-fcf74aa2cfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:40.710442 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-852fdda5-b71c-4508-ae87-fcf74aa2cfe8 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:40.711950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-852fdda5-b71c-4508-ae87-fcf74aa2cfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:40.806762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 238/457] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:40.843884 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397e179-086e-4f2c-83cb-52c757be5ed2 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:40.960016 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397e179-086e-4f2c-83cb-52c757be5ed2 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:40.995000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d4d60-d5bf-4c9d-b586-b0d0c77f91e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:40.995811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4d60-d5bf-4c9d-b586-b0d0c77f91e5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:40.996102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4d60-d5bf-4c9d-b586-b0d0c77f91e5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:40.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d4d60-d5bf-4c9d-b586-b0d0c77f91e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.007421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d4d60-d5bf-4c9d-b586-b0d0c77f91e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:41.008913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d4d60-d5bf-4c9d-b586-b0d0c77f91e5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-6d2d4d60-d5bf-4c9d-b586-b0d0c77f91e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:41.026588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:41.027549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:41.028465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:41.028465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:41.028465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:41.028465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:41.028465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 220/458] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:41.033722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fdda5-b71c-4508-ae87-fcf74aa2cfe8 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:41.167718 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a67fc-f437-494f-8002-13196534d090 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.168065 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a67fc-f437-494f-8002-13196534d090 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.168297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a67fc-f437-494f-8002-13196534d090 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.169975 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 221/459] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:41.178453 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf30601-d38f-4692-8e4e-61dcec790550 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.178732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf30601-d38f-4692-8e4e-61dcec790550 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.179016 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf30601-d38f-4692-8e4e-61dcec790550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.204373 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-6cf30601-d38f-4692-8e4e-61dcec790550 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:41.205701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-6cf30601-d38f-4692-8e4e-61dcec790550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:41.218391 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fdda5-b71c-4508-ae87-fcf74aa2cfe8 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:41.254788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:41.255456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.255735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.256009 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.256738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:41.257059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:41.261277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:41.300748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 239/460] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:41.306660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:41.337184 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 222/461] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:41.338738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff8fca-0ae1-4556-9035-3935dafae8b2 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.339031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff8fca-0ae1-4556-9035-3935dafae8b2 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.339316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff8fca-0ae1-4556-9035-3935dafae8b2 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.340675 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 223/462] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:41.359007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f951-51b2-4780-8b43-bd53ba3e0922 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.359309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f951-51b2-4780-8b43-bd53ba3e0922 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.359575 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f951-51b2-4780-8b43-bd53ba3e0922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.379392 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-4772f951-51b2-4780-8b43-bd53ba3e0922 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:41.380598 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-4772f951-51b2-4780-8b43-bd53ba3e0922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:41.411747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4a504-7d56-43da-a981-55b017dee80e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:41.446515 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224311b-19b5-4719-a63a-12f926877203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:41.447373 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224311b-19b5-4719-a63a-12f926877203 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.447656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224311b-19b5-4719-a63a-12f926877203 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.447931 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224311b-19b5-4719-a63a-12f926877203 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.448697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 224/463] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:41.449595 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772f951-51b2-4780-8b43-bd53ba3e0922 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:41.457164 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224311b-19b5-4719-a63a-12f926877203 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:41.458725 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224311b-19b5-4719-a63a-12f926877203 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:41.490233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 240/464] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:41 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:41.491415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a8c7c-b9cf-43c1-a60d-b2abf7905fdb None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.491691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a8c7c-b9cf-43c1-a60d-b2abf7905fdb None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.492049 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a8c7c-b9cf-43c1-a60d-b2abf7905fdb None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.493565 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 241/465] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:41.494285 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a8c7c-b9cf-43c1-a60d-b2abf7905fdb None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:41.555878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772f951-51b2-4780-8b43-bd53ba3e0922 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:41.593699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f86ae-7ba7-4578-8f3d-d08c0b590bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:41.594526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f86ae-7ba7-4578-8f3d-d08c0b590bb0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.594813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f86ae-7ba7-4578-8f3d-d08c0b590bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.595103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f86ae-7ba7-4578-8f3d-d08c0b590bb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.604911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a8c7c-b9cf-43c1-a60d-b2abf7905fdb None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:41.606941 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f86ae-7ba7-4578-8f3d-d08c0b590bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:41.608702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f86ae-7ba7-4578-8f3d-d08c0b590bb0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5f5f86ae-7ba7-4578-8f3d-d08c0b590bb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:41.627791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:41.630332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:41.632549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:41.632549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:41.632549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:41.632549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:41.632549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 225/466] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:41.632549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f16733-163a-469d-82a9-13e494c25db3 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.632549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f16733-163a-469d-82a9-13e494c25db3 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.632549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f16733-163a-469d-82a9-13e494c25db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.652647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:41.653325 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-a4f16733-163a-469d-82a9-13e494c25db3 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:41.653595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.653881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.654156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.654933 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-a4f16733-163a-469d-82a9-13e494c25db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:41.669687 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:41.671916 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:41.693192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:41.695240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:41.697248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:41.697248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:41.697248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:41.697248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:41.697248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 242/467] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:41.697248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:41.729928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 226/468] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:41.731423 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f16733-163a-469d-82a9-13e494c25db3 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:41.813876 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e719346-625a-4a9a-ba3f-9afbf133b024 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:41.845916 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f16733-163a-469d-82a9-13e494c25db3 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:41.852678 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:41.853383 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.853668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.853944 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.854680 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:41.854986 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:41.856368 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:41.869343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 243/469] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:41.870196 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:41.901891 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:41.902557 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:41.902839 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:41.903111 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:41.903837 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:41.904141 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:41.905685 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:41.917557 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 227/470] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:41.918159 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:41.997693 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b9ba1-e762-481c-8b9c-66f82ae1af72 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:42.022783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03709a96-a936-468d-b45a-8de9948c1e2f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:42.044659 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:42.045992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.046495 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.047132 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:42.057924 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca0ca8-ecd8-4725-bea8-acf7f476e04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:42.058601 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca0ca8-ecd8-4725-bea8-acf7f476e04c None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.059123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca0ca8-ecd8-4725-bea8-acf7f476e04c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.059237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca0ca8-ecd8-4725-bea8-acf7f476e04c None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:42.059897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca0ca8-ecd8-4725-bea8-acf7f476e04c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:42.061439 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca0ca8-ecd8-4725-bea8-acf7f476e04c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:42.061687 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:42.063460 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:42.084630 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 228/471] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:41 2022] POST /identity/v3/users => generated 248 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:42.093066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:42.094085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:42.095012 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:42.095012 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:42.095012 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:42.095012 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:42.095254 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca0ca8-ecd8-4725-bea8-acf7f476e04c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:42.101265 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 244/472] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:42.110363 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:42.236775 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca0ca8-ecd8-4725-bea8-acf7f476e04c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:42.292945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc11f2a-f402-4773-9623-d159d08bb55c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:42.293891 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f1e46-da02-488e-9a95-9e5941276e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:42.294609 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f1e46-da02-488e-9a95-9e5941276e4e None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.294904 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f1e46-da02-488e-9a95-9e5941276e4e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.295190 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f1e46-da02-488e-9a95-9e5941276e4e None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:42.295948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f1e46-da02-488e-9a95-9e5941276e4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:42.297535 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f1e46-da02-488e-9a95-9e5941276e4e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:42.338602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaf49b-179e-4ce4-8013-675cb373449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:42.339203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf49b-179e-4ce4-8013-675cb373449c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.339545 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf49b-179e-4ce4-8013-675cb373449c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.339975 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf49b-179e-4ce4-8013-675cb373449c None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:42.340856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaf49b-179e-4ce4-8013-675cb373449c None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:42.341267 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaf49b-179e-4ce4-8013-675cb373449c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:42.343396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaf49b-179e-4ce4-8013-675cb373449c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:42.356887 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 229/473] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:42 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:42.365871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 245/474] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:42.374239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f1e46-da02-488e-9a95-9e5941276e4e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:42.476781 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb0f2d-1edd-4c42-92c2-5cbff80fc3c0 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.477221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb0f2d-1edd-4c42-92c2-5cbff80fc3c0 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.477506 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb0f2d-1edd-4c42-92c2-5cbff80fc3c0 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:42.478915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 246/475] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:42.489255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c798ce2-9ad5-497b-a0b6-4d66fd681c09 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.489543 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c798ce2-9ad5-497b-a0b6-4d66fd681c09 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.489819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c798ce2-9ad5-497b-a0b6-4d66fd681c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:42.501435 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f1e46-da02-488e-9a95-9e5941276e4e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:42.513721 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-6c798ce2-9ad5-497b-a0b6-4d66fd681c09 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:42.515599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-6c798ce2-9ad5-497b-a0b6-4d66fd681c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:42.541504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70769d80-8d6d-4295-9b28-d8542ee126b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:42.542526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70769d80-8d6d-4295-9b28-d8542ee126b1 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.543038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70769d80-8d6d-4295-9b28-d8542ee126b1 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.543519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70769d80-8d6d-4295-9b28-d8542ee126b1 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:42.544539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70769d80-8d6d-4295-9b28-d8542ee126b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:42.546321 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70769d80-8d6d-4295-9b28-d8542ee126b1 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:42.573632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 230/476] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:42 2022] POST /identity/v3/users => generated 249 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:42.604043 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 247/477] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:42.629251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70769d80-8d6d-4295-9b28-d8542ee126b1 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:42.743368 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70769d80-8d6d-4295-9b28-d8542ee126b1 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:42.780973 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:42.782004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.782487 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.782924 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:42.783868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:42.786038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:42.793930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.sql.core [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:28:42.793930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 15:28:42.793930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [parameters: {'id': 'a42852ecbb5745a294f50c236d019dd0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 15:28:42.793930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=206281) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application conn.query(q) May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application result.read() May 09 15:28:42.812155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return ref.to_dict() May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:28:42.813426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application next(self.gen) May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application yield resource May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application next(self.gen) May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise self.value May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application session.commit() May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:28:42.814705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self.session.flush() May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application compat.raise_( May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:28:42.815783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:28:42.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application conn.query(q) May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application result.read() May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:28:42.817669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [parameters: {'id': 'a42852ecbb5745a294f50c236d019dd0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:42.818541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 15:28:42.819481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:42.822355 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 231/478] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:28:42 2022] POST /identity/v3/roles => generated 144 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:42.828629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c798ce2-9ad5-497b-a0b6-4d66fd681c09 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:42.947969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c798ce2-9ad5-497b-a0b6-4d66fd681c09 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:42.986907 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfef7b1-fa82-4343-844b-7c2a39634c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:42.987774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef7b1-fa82-4343-844b-7c2a39634c7a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:42.988062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef7b1-fa82-4343-844b-7c2a39634c7a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:42.988340 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef7b1-fa82-4343-844b-7c2a39634c7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:43.002254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfef7b1-fa82-4343-844b-7c2a39634c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-abfef7b1-fa82-4343-844b-7c2a39634c7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:43.010431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:43.011420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:43.012322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 248/479] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:28:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:43.017932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:43.142161 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99bfec-cb7b-447b-a1b8-c84885d4d784 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:43.177306 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66b06a-74ee-43e5-9328-f5bc81697896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:43.178371 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b06a-74ee-43e5-9328-f5bc81697896 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:43.178873 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b06a-74ee-43e5-9328-f5bc81697896 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:43.179387 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b06a-74ee-43e5-9328-f5bc81697896 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:43.180433 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66b06a-74ee-43e5-9328-f5bc81697896 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:43.180995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66b06a-74ee-43e5-9328-f5bc81697896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:43.182613 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66b06a-74ee-43e5-9328-f5bc81697896 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:43.191780 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 232/480] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:44.083632 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00994ae-1b4f-4e16-8b1b-78bad02a2746 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:44.084707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00994ae-1b4f-4e16-8b1b-78bad02a2746 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:44.085250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00994ae-1b4f-4e16-8b1b-78bad02a2746 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:44.087224 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 249/481] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:44.100544 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d95fd-cd7c-47ab-8f4a-3aab11326205 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:44.101030 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d95fd-cd7c-47ab-8f4a-3aab11326205 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:44.101158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d95fd-cd7c-47ab-8f4a-3aab11326205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:44.124637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-328d95fd-cd7c-47ab-8f4a-3aab11326205 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:44.125865 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-328d95fd-cd7c-47ab-8f4a-3aab11326205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:44.220089 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 233/482] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:44.255726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00994ae-1b4f-4e16-8b1b-78bad02a2746 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:44.395359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00994ae-1b4f-4e16-8b1b-78bad02a2746 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:44.437653 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b9e3e-eeac-401e-be0a-362c5b9e4e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:44.438459 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b9e3e-eeac-401e-be0a-362c5b9e4e81 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:44.438746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b9e3e-eeac-401e-be0a-362c5b9e4e81 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:44.439024 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b9e3e-eeac-401e-be0a-362c5b9e4e81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:44.453982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b9e3e-eeac-401e-be0a-362c5b9e4e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-fd2b9e3e-eeac-401e-be0a-362c5b9e4e81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:44.462388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:44.463407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:44.464290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 250/483] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:44.469400 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d95fd-cd7c-47ab-8f4a-3aab11326205 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:44.603507 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d95fd-cd7c-47ab-8f4a-3aab11326205 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:44.638893 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:44.639572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:44.639854 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:44.640131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:44.640890 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:44.641221 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:44.642589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:44.651284 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 234/484] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:44.656790 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b9e3e-eeac-401e-be0a-362c5b9e4e81 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:44.809401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b9e3e-eeac-401e-be0a-362c5b9e4e81 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:44.851698 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf81fea-d7d5-41f7-a623-bd5937672c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:44.852506 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf81fea-d7d5-41f7-a623-bd5937672c29 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:44.852795 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf81fea-d7d5-41f7-a623-bd5937672c29 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:44.853090 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf81fea-d7d5-41f7-a623-bd5937672c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:44.864959 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf81fea-d7d5-41f7-a623-bd5937672c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:44.866468 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf81fea-d7d5-41f7-a623-bd5937672c29 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-bbf81fea-d7d5-41f7-a623-bd5937672c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:44.894856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:44.897200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:44.899477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:44.899477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:44.899477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:44.899477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:44.899477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 251/485] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:44.901799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:44.993809 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f700da0-43dc-49c7-9871-bd18f237455b None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:44.994172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f700da0-43dc-49c7-9871-bd18f237455b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:44.994345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f700da0-43dc-49c7-9871-bd18f237455b None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:44.995996 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 252/486] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:45.001097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec8fa-a2e1-432f-8c4d-9d33763fcc8e None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.001810 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec8fa-a2e1-432f-8c4d-9d33763fcc8e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.002439 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec8fa-a2e1-432f-8c4d-9d33763fcc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.042085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-6f7ec8fa-a2e1-432f-8c4d-9d33763fcc8e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:45.043671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-6f7ec8fa-a2e1-432f-8c4d-9d33763fcc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:45.054384 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2b289-14be-4286-ac1e-84369ee7fe59 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:45.107784 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7c1a5-9d55-4aea-84b1-165234463c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:45.108930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7c1a5-9d55-4aea-84b1-165234463c84 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.109458 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7c1a5-9d55-4aea-84b1-165234463c84 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.109956 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7c1a5-9d55-4aea-84b1-165234463c84 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.111309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7c1a5-9d55-4aea-84b1-165234463c84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:45.111864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7c1a5-9d55-4aea-84b1-165234463c84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:45.118233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7c1a5-9d55-4aea-84b1-165234463c84 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:45.141188 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 235/487] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:45.143380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810f57-4c0d-4809-9673-d10a7207a1b7 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.143904 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810f57-4c0d-4809-9673-d10a7207a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.144364 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810f57-4c0d-4809-9673-d10a7207a1b7 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.146607 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 236/488] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:45.150326 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810f57-4c0d-4809-9673-d10a7207a1b7 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:45.198200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 253/489] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:45.200386 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf0c0-2a09-4d3e-a1df-159debfb9fcd None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.200890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf0c0-2a09-4d3e-a1df-159debfb9fcd None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.209477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf0c0-2a09-4d3e-a1df-159debfb9fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.254689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-db5cf0c0-2a09-4d3e-a1df-159debfb9fcd None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:45.255318 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810f57-4c0d-4809-9673-d10a7207a1b7 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:45.256675 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-db5cf0c0-2a09-4d3e-a1df-159debfb9fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:45.290543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:45.291339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.292025 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.292305 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] PATH_INFO: `/v3/users/nova` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.300323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:45.301797 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:45.322310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:45.324811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 237/490] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:28:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:45.327266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:45.387263 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 254/491] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:45.389453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cf0c0-2a09-4d3e-a1df-159debfb9fcd None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:45.460513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5c47c-bda4-460a-9fc6-c5f70005fcec None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:45.496871 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392518b8-e2a0-4161-bd85-a79513142c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:45.497741 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392518b8-e2a0-4161-bd85-a79513142c5a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.498028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392518b8-e2a0-4161-bd85-a79513142c5a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.498479 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392518b8-e2a0-4161-bd85-a79513142c5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.508960 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392518b8-e2a0-4161-bd85-a79513142c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-392518b8-e2a0-4161-bd85-a79513142c5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:45.517339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:45.520174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 238/492] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:45.522418 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392518b8-e2a0-4161-bd85-a79513142c5a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:45.526196 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cf0c0-2a09-4d3e-a1df-159debfb9fcd None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:45.560120 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74b978-d55b-4290-8789-788ace1e9a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:45.560796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74b978-d55b-4290-8789-788ace1e9a51 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.561114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74b978-d55b-4290-8789-788ace1e9a51 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.561395 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74b978-d55b-4290-8789-788ace1e9a51 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.562251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74b978-d55b-4290-8789-788ace1e9a51 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:45.562562 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74b978-d55b-4290-8789-788ace1e9a51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:45.564022 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74b978-d55b-4290-8789-788ace1e9a51 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:45.604933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 255/493] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:28:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:45.604933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424779fc-e070-47ed-a69a-9aee63ad2628 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.604933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424779fc-e070-47ed-a69a-9aee63ad2628 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.604933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424779fc-e070-47ed-a69a-9aee63ad2628 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.608340 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 256/494] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:45.608999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424779fc-e070-47ed-a69a-9aee63ad2628 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:45.627604 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392518b8-e2a0-4161-bd85-a79513142c5a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:45.661119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7825ed6-7078-41d6-ad25-7e61a764c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:45.661913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825ed6-7078-41d6-ad25-7e61a764c9dd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.662197 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825ed6-7078-41d6-ad25-7e61a764c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.662470 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7825ed6-7078-41d6-ad25-7e61a764c9dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.672292 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7825ed6-7078-41d6-ad25-7e61a764c9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-b7825ed6-7078-41d6-ad25-7e61a764c9dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:45.679983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:45.680980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:45.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:45.686499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 239/495] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:45.686499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7825ed6-7078-41d6-ad25-7e61a764c9dd None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:45.712058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424779fc-e070-47ed-a69a-9aee63ad2628 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:45.749121 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01c6bd-f69e-4a6c-956f-99b263a1a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:45.749802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01c6bd-f69e-4a6c-956f-99b263a1a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.750087 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01c6bd-f69e-4a6c-956f-99b263a1a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.750372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01c6bd-f69e-4a6c-956f-99b263a1a1c6 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.751137 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01c6bd-f69e-4a6c-956f-99b263a1a1c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:45.751448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01c6bd-f69e-4a6c-956f-99b263a1a1c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:45.752853 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01c6bd-f69e-4a6c-956f-99b263a1a1c6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:45.765318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 257/496] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:45.765318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559fa9e-d6da-429a-9a86-58a2f9816bfc None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.765318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559fa9e-d6da-429a-9a86-58a2f9816bfc None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.765318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559fa9e-d6da-429a-9a86-58a2f9816bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.778771 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-4559fa9e-d6da-429a-9a86-58a2f9816bfc None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:45.779961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-4559fa9e-d6da-429a-9a86-58a2f9816bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:45.787615 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7825ed6-7078-41d6-ad25-7e61a764c9dd None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:45.821172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea6f8a-d607-46ca-ac68-bec318ebc418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:45.821835 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea6f8a-d607-46ca-ac68-bec318ebc418 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.822122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea6f8a-d607-46ca-ac68-bec318ebc418 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.822406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea6f8a-d607-46ca-ac68-bec318ebc418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.855050 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 258/497] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:45.856184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559fa9e-d6da-429a-9a86-58a2f9816bfc None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:45.903347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea6f8a-d607-46ca-ac68-bec318ebc418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:45.905250 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea6f8a-d607-46ca-ac68-bec318ebc418 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:45.953201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559fa9e-d6da-429a-9a86-58a2f9816bfc None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:45.988685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:45.989312 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:45.989622 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:45.989864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:45.990610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] RBAC: Adding query filter params (name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:45.990913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:45.992288 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.000250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 259/498] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:46.001073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.002216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 240/499] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:46.003837 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea6f8a-d607-46ca-ac68-bec318ebc418 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.106315 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3048-c826-4e63-b69e-fd4115eecaa0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.109794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea6f8a-d607-46ca-ac68-bec318ebc418 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.139275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.140286 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.140685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.141053 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.142913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef150837-23c1-41f7-98c5-2827745450ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.144237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef150837-23c1-41f7-98c5-2827745450ba None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.144522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef150837-23c1-41f7-98c5-2827745450ba None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.144798 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef150837-23c1-41f7-98c5-2827745450ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.152114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.153719 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.155686 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef150837-23c1-41f7-98c5-2827745450ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-ef150837-23c1-41f7-98c5-2827745450ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:46.164108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:46.165144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 241/500] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:46.171178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef150837-23c1-41f7-98c5-2827745450ba None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:46.171982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:46.172834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:46.173813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:46.173813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:46.173813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:46.173813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:46.173813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 260/501] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:46.173813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.273065 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef150837-23c1-41f7-98c5-2827745450ba None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.282592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d04dfa-0092-4e7f-9d1e-f38469b189a4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.308672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.309576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.309866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.310163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.317226 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.318009 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.318287 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.318555 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.321939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.323462 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.329955 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.332036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:46.342813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:46.343825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:46.344766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:46.344766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:46.344766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:46.344766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:46.344766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 242/502] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:46.345574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:46.352281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:46.353360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:46.354294 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:46.354294 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:46.354294 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:46.354294 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:46.354294 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 261/503] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:46.360125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.452216 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d39c3-70d8-40ff-b7ec-9f9cca1b4028 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.453926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34b903-ea11-4edd-8a7c-e305e4a5c415 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.486048 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.486728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.487020 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.487306 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.488175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.488620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:46.488846 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.488957 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.489164 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.489444 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.490172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:46.490373 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.490481 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.491852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.502837 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 243/504] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:46.502837 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.509075 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 262/505] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:46.513416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.599109 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c07da-c1bf-40a2-85ce-a165c7d9895a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.604905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4335d-ed91-42f5-8b52-4c85882770b8 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.631374 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.632038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.632329 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.632613 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.633379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:46.633692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.635067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.637726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.638376 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.638656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.638929 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.639672 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:46.639976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.641365 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.646866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 244/506] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:46.647658 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.652842 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 263/507] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:46.653556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.746756 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6e665-9add-49c9-9bcd-3542066f0cde None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.750400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072a606-7def-4f69-a0dd-3559df109678 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.789302 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b520b3-bd76-432d-91d0-cd161b1ee522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.789659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.790092 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b520b3-bd76-432d-91d0-cd161b1ee522 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.790379 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b520b3-bd76-432d-91d0-cd161b1ee522 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.790476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.790651 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b520b3-bd76-432d-91d0-cd161b1ee522 None admin] PATH_INFO: `/v3/users/glance` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.790762 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.790978 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.797941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b520b3-bd76-432d-91d0-cd161b1ee522 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.799385 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b520b3-bd76-432d-91d0-cd161b1ee522 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.801371 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.802809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-62b520b3-bd76-432d-91d0-cd161b1ee522 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:46.815149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:46.816713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:46.817764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 264/508] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:46.818471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170aed7-5f69-4a4a-bf18-1cbba9b59cdb None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.818471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170aed7-5f69-4a4a-bf18-1cbba9b59cdb None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.819119 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170aed7-5f69-4a4a-bf18-1cbba9b59cdb None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.820219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 265/509] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:46.820930 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170aed7-5f69-4a4a-bf18-1cbba9b59cdb None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:46.821236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:46.822600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:46.823981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:46.823981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:46.823981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:46.823981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:46.823981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 245/510] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:46.824499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.918425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee22d7-f870-428d-8a14-cd6fcd6003a5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.923244 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170aed7-5f69-4a4a-bf18-1cbba9b59cdb None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:46.951158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.951949 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.952240 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.952521 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.955234 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca4f49-f960-4c4e-b850-22468bbb5c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:46.956009 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca4f49-f960-4c4e-b850-22468bbb5c50 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.956317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca4f49-f960-4c4e-b850-22468bbb5c50 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.956592 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca4f49-f960-4c4e-b850-22468bbb5c50 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.959351 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.960808 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.963987 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca4f49-f960-4c4e-b850-22468bbb5c50 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:46.965458 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca4f49-f960-4c4e-b850-22468bbb5c50 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:46.975205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:46.976213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 246/511] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:46.977404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-55ca4f49-f960-4c4e-b850-22468bbb5c50 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:46.979199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:46.980127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 266/512] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:46.981855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e41fa-a671-43b1-93e4-2bafbc1de640 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:46.982955 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e41fa-a671-43b1-93e4-2bafbc1de640 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:46.982955 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e41fa-a671-43b1-93e4-2bafbc1de640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:46.999042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-068e41fa-a671-43b1-93e4-2bafbc1de640 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:47.000245 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-068e41fa-a671-43b1-93e4-2bafbc1de640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:47.063586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 267/513] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:47.064412 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e41fa-a671-43b1-93e4-2bafbc1de640 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.098451 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06680d3e-5270-4c23-9169-ccf8d5f8e06b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.139749 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.140429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.140710 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.140990 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.142656 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:47.143164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.145313 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.172182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e41fa-a671-43b1-93e4-2bafbc1de640 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.182145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 247/514] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:28:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:47.182994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.204867 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.205566 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.205853 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.206161 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.207450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:47.207757 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.209179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.219952 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 268/515] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:47.220728 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.280635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27506f-446e-4bc5-bc10-da1e6cc019c6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.312425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.313112 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.313402 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.313680 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.313986 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a09753-d689-4d1d-a5a4-e2ee94bc0604 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.314477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:47.314789 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.316806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.334437 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 248/516] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:47.335239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.346463 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.347181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.347445 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.347717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.348551 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:47.348859 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.350334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.367043 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 269/517] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:47.367831 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.433536 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a81e62-2af1-4bb6-85ac-edbfa2dfff80 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.461327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720a104-7106-4821-ab8c-2e3a6bc46566 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.465992 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.466780 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.467062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.467338 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.477626 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.479090 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.493235 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ecc3c-07ee-4331-bd17-c2bb8eea09d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.493866 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ecc3c-07ee-4331-bd17-c2bb8eea09d6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.494148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ecc3c-07ee-4331-bd17-c2bb8eea09d6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.494421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ecc3c-07ee-4331-bd17-c2bb8eea09d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:47.496446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:47.497590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:47.498948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:47.498948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:47.498948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:47.498948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:47.498948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 249/518] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:47.498948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.560071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ecc3c-07ee-4331-bd17-c2bb8eea09d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.561623 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ecc3c-07ee-4331-bd17-c2bb8eea09d6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.592164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33519278-3a13-4abf-a4c9-22b7ba4954b6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.624139 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.624916 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.625230 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.625509 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.633347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.634786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:47.648584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:47.649700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 250/519] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:47.650691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.661183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 270/520] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:47.661183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ecc3c-07ee-4331-bd17-c2bb8eea09d6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.746242 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f911d21-eaa0-48b8-bc2e-34dd34c0d0c7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.751106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ecc3c-07ee-4331-bd17-c2bb8eea09d6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.777947 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.778587 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.778868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.779146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.779972 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=17730dd371184663b2c0eb091cc668a3) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:47.780285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.781751 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.783300 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842c6d-b953-4a54-b435-ad5aa96570e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.783938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842c6d-b953-4a54-b435-ad5aa96570e2 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.784217 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842c6d-b953-4a54-b435-ad5aa96570e2 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.784487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842c6d-b953-4a54-b435-ad5aa96570e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.786981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 251/521] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/role_assignments?user.id=17730dd371184663b2c0eb091cc668a3&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:47.787814 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.856643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27842c6d-b953-4a54-b435-ad5aa96570e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.858938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27842c6d-b953-4a54-b435-ad5aa96570e2 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.889852 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a203d-040e-4c17-bf1c-f66dba1cefbf None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:47.933061 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:47.933759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:47.934052 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:47.934336 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:47.935076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:47.935701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:47.936900 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:47.949715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 252/522] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:47.952412 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:47.974133 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 271/523] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:47.975318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842c6d-b953-4a54-b435-ad5aa96570e2 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.060311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d2d40-531c-4bce-910c-8d886b319274 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.074414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842c6d-b953-4a54-b435-ad5aa96570e2 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.099905 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.100586 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.100932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.101187 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.102222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:48.102537 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.103980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.111072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.111867 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.112273 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.112981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.126927 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 253/524] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:48.127107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.127221 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.127303 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:48.144669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:48.145717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:48.146673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:48.146673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:48.146673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:48.146673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:48.150440 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 272/525] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:48.151048 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.226085 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddd69e-6157-4e98-8231-b2f4b90a194f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.249953 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f22895-663d-4b7f-ab35-702fdf53e44f None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.259214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e019c-c31b-49ca-9dd3-c561196cead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.259892 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e019c-c31b-49ca-9dd3-c561196cead2 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.260180 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e019c-c31b-49ca-9dd3-c561196cead2 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.260467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e019c-c31b-49ca-9dd3-c561196cead2 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.261213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e019c-c31b-49ca-9dd3-c561196cead2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.262676 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e019c-c31b-49ca-9dd3-c561196cead2 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.293077 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 254/526] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:28:48 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:28:48.293077 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e019c-c31b-49ca-9dd3-c561196cead2 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.294511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.295416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.295766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.296103 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.308085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.309713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:48.332671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:48.334114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:48.335540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:48.335540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:48.335540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:48.335540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:48.338684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 273/527] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:48.339615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.405160 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e019c-c31b-49ca-9dd3-c561196cead2 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.441410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cbe91-a0b2-4829-ae8c-433361a3922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.442073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cbe91-a0b2-4829-ae8c-433361a3922b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.442355 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cbe91-a0b2-4829-ae8c-433361a3922b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.442628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cbe91-a0b2-4829-ae8c-433361a3922b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.451799 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500aca6-55e5-4ce1-a3c1-0673abbe9ebf None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.488916 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.489640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.489928 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.490208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.490949 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:48.491264 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.492651 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.506792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 274/528] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:48.507657 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.520930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812cbe91-a0b2-4829-ae8c-433361a3922b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.522454 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812cbe91-a0b2-4829-ae8c-433361a3922b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.627963 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 255/529] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:48.629610 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cbe91-a0b2-4829-ae8c-433361a3922b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.633986 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d270c7-201d-4a8c-9ab4-80e6b7103908 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.676868 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.677939 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.678228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.678508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.679923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:48.680243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.681860 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.702812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 275/530] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:48.703905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.738705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cbe91-a0b2-4829-ae8c-433361a3922b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.773112 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.773913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.774192 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.774466 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.781254 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.782700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:48.800424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:48.801421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:48.802461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:48.802461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:48.802461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:48.802461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:48.802461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:48.802461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:48.802461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:48.802461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:48.807276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f1f23-c137-463c-8e87-6a7f9638dfd2 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.807756 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 256/531] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:48.809443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.841931 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.842729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.843129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.843411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.853408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.854945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:48.872587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:48.874961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:48.877411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:48.877411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:48.877411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:48.877411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:48.877411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 276/532] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:48.877411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:48.931777 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e6971-dc15-4ecd-965a-d80fd736d601 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:48.975357 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:48.976803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:48.977253 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:48.977600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:48.987094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:48.990183 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:48.998712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f03c3f-e1b4-496e-8aaf-a5a7f3c0bfae None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:49.009213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:49.010369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:49.011546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:49.011546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:49.011546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:49.011546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:49.011546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:49.011546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:49.011546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:49.011546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:49.017112 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 257/533] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:49.018100 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:49.043347 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:49.044352 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:49.044739 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:49.045256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:49.046163 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:49.046555 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:49.048077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:49.066628 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 277/534] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:49.067680 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:49.137909 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53d6f4-8b0c-421e-aafc-b5c8fe70cd10 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:49.173730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:49.174714 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:49.175247 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:49.175719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:49.176267 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceea3c6-dad0-44ee-9347-f38d8898ab5b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:49.176878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:49.177415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:49.178998 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:49.195514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 258/535] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:49.196732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:49.211318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:49.211959 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:49.212247 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:49.212527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:49.213362 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:49.213677 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:49.215120 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:49.227943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 278/536] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:49.233829 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:49.304067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ecfe5-0565-4fd1-a45e-f14fd3bc25d5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:49.334754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146abd6-bd3f-43eb-aacc-a66788266116 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:49.340475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:49.341428 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:49.341935 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:49.342457 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:49.343529 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=4e125e49ac664c749fe6eede9531920d) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:49.344063 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:49.345749 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:49.351875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 259/537] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:49 2022] GET /identity/v3/role_assignments?user.id=4e125e49ac664c749fe6eede9531920d&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:49.353084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:49.369411 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:49.370251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:49.370545 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:49.370826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:49.378664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:49.380267 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:49.398723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:49.399880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:49.401499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 279/538] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:49.406108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:49.468916 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4e9d4-88d9-40a3-a6e1-21ac72c3e9ae None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:49.505177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:49.506234 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:49.506739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:49.507236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:49.508340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=365477e05c1f48beac7f88e6a0707bd6) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:49.508905 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:49.510668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:49.512431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898c815-807c-43d4-8699-82165d0f5a06 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:49.518006 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 260/539] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:49 2022] GET /identity/v3/role_assignments?user.id=365477e05c1f48beac7f88e6a0707bd6&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:49.586027 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e36fa5-c9f1-4e21-9b1f-0a6a8eefc007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:49.586745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e36fa5-c9f1-4e21-9b1f-0a6a8eefc007 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:49.587036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e36fa5-c9f1-4e21-9b1f-0a6a8eefc007 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:49.587318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e36fa5-c9f1-4e21-9b1f-0a6a8eefc007 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:49.588170 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e36fa5-c9f1-4e21-9b1f-0a6a8eefc007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:49.588487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e36fa5-c9f1-4e21-9b1f-0a6a8eefc007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:49.590064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e36fa5-c9f1-4e21-9b1f-0a6a8eefc007 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:49.604626 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 280/540] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:49.610664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:49.739649 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf664d39-62e8-4616-9c88-c97569787840 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:49.789343 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b2061-d16a-4021-85eb-d6a47bea56d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:49.790018 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b2061-d16a-4021-85eb-d6a47bea56d0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:49.790308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b2061-d16a-4021-85eb-d6a47bea56d0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:49.790585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b2061-d16a-4021-85eb-d6a47bea56d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:49.791429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b2061-d16a-4021-85eb-d6a47bea56d0 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=5382d7219c244430ae6f37d6b32e7d8a) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:49.791741 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b2061-d16a-4021-85eb-d6a47bea56d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:49.793202 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b2061-d16a-4021-85eb-d6a47bea56d0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:49.802157 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 261/541] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:49 2022] GET /identity/v3/role_assignments?user.id=5382d7219c244430ae6f37d6b32e7d8a&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:50.197859 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59d80-05fe-4c8f-a7b9-3ca794d21a18 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:50.198227 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59d80-05fe-4c8f-a7b9-3ca794d21a18 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:50.198425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc59d80-05fe-4c8f-a7b9-3ca794d21a18 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:50.200067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 281/542] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:50.205625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e76ca-a5a9-4da8-98cf-3bf5b9e07013 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:50.205926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e76ca-a5a9-4da8-98cf-3bf5b9e07013 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:50.206314 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e76ca-a5a9-4da8-98cf-3bf5b9e07013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:50.231577 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-848e76ca-a5a9-4da8-98cf-3bf5b9e07013 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:50.233084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-848e76ca-a5a9-4da8-98cf-3bf5b9e07013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:50.332993 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 262/543] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:50.367885 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc59d80-05fe-4c8f-a7b9-3ca794d21a18 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:50.506815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc59d80-05fe-4c8f-a7b9-3ca794d21a18 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:50.559026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c5979-419a-452b-995f-7bd639980e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:50.559840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c5979-419a-452b-995f-7bd639980e9d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:50.560206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c5979-419a-452b-995f-7bd639980e9d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:50.560489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c5979-419a-452b-995f-7bd639980e9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:50.582149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c5979-419a-452b-995f-7bd639980e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-010c5979-419a-452b-995f-7bd639980e9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:50.590000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:50.591011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:50.592493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 282/544] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:50.597659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e76ca-a5a9-4da8-98cf-3bf5b9e07013 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:50.750460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e76ca-a5a9-4da8-98cf-3bf5b9e07013 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:50.799146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be845e-0a61-4c76-a2f5-4d8a397d1842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:50.799901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be845e-0a61-4c76-a2f5-4d8a397d1842 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:50.800185 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be845e-0a61-4c76-a2f5-4d8a397d1842 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:50.800460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be845e-0a61-4c76-a2f5-4d8a397d1842 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:50.801240 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be845e-0a61-4c76-a2f5-4d8a397d1842 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:50.801552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be845e-0a61-4c76-a2f5-4d8a397d1842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:50.802924 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be845e-0a61-4c76-a2f5-4d8a397d1842 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:50.816912 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 263/545] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:50.822232 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c5979-419a-452b-995f-7bd639980e9d None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:50.944896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c5979-419a-452b-995f-7bd639980e9d None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:50.984420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:50.985630 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:50.986131 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:50.986621 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:50.998111 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b2675-cdaa-4e9d-972b-e313f6247e79 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:50.998398 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b2675-cdaa-4e9d-972b-e313f6247e79 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:50.998683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b2675-cdaa-4e9d-972b-e313f6247e79 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:51.000351 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 264/546] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:51.006053 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286b350-b5aa-4cda-b66b-29b436b83f4c None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:51.006331 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286b350-b5aa-4cda-b66b-29b436b83f4c None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:51.006615 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286b350-b5aa-4cda-b66b-29b436b83f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:51.008619 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:51.010567 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:51.040527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:51.041586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:51.042538 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:51.042538 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:51.042538 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:51.042538 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:51.044222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-f286b350-b5aa-4cda-b66b-29b436b83f4c None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:51.045969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-f286b350-b5aa-4cda-b66b-29b436b83f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:51.050977 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 283/547] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:51.055189 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:51.196264 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aba1b-2931-42e6-88d6-7b1ac1d35442 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:51.207453 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 265/548] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:51.233379 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:51.234408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:51.234926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:51.235417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:51.236368 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:51.236891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:51.239032 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:51.244147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286b350-b5aa-4cda-b66b-29b436b83f4c None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:51.251639 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 284/549] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:51.263826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:51.363979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1878359-fcb3-44aa-bce9-f0d9d67e1356 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:51.398450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:51.399264 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:51.399553 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:51.399835 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] PATH_INFO: `/v3/users/nova` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:51.407969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:51.409538 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:51.425649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:51.426815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:51.427832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 285/550] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:28:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:51.434125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:51.457186 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286b350-b5aa-4cda-b66b-29b436b83f4c None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:51.492296 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04d51-eb81-4a6c-9e1a-0411eba0bf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:51.493142 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04d51-eb81-4a6c-9e1a-0411eba0bf85 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:51.493428 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04d51-eb81-4a6c-9e1a-0411eba0bf85 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:51.493707 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec04d51-eb81-4a6c-9e1a-0411eba0bf85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:51.504644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec04d51-eb81-4a6c-9e1a-0411eba0bf85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-aec04d51-eb81-4a6c-9e1a-0411eba0bf85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:51.512394 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:51.514440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:51.515621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 266/551] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:51.519640 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04d51-eb81-4a6c-9e1a-0411eba0bf85 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:51.544969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2ec5d-4ade-4b2c-89b4-2289093c7152 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:51.592580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:51.593311 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:51.593600 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:51.593908 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:51.594754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:51.595076 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:51.596563 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:51.638108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 286/552] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:28:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:51.644647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:51.729462 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec04d51-eb81-4a6c-9e1a-0411eba0bf85 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:51.796059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:51.796659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:51.796946 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:51.797362 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:51.798024 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:51.798342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:51.799741 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:51.817007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 267/553] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:51.825994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:51.839821 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9976d-e54f-41b1-bd3d-c3cf955ead37 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:51.896405 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:51.897239 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:51.897573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:51.897779 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:51.918409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:51.919908 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:51.931195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606b82-a52b-47db-b7d7-c2145876afaa None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:51.949643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:51.951843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:51.954219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:51.954219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:51.954219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:51.954219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:51.957485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 287/554] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:51.962534 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:51.976338 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:51.977135 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:51.977419 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:51.977694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.001115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.002454 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:52.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:52.034084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:52.036898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:52.036898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:52.036898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:52.036898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:52.042336 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 268/555] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:52.043235 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.122750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcbfb8-18fe-4f14-b1e9-15f41f183bbd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.157608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770372f-eec5-424a-9046-a81b265df1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.158276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770372f-eec5-424a-9046-a81b265df1f2 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.158557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770372f-eec5-424a-9046-a81b265df1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.158832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770372f-eec5-424a-9046-a81b265df1f2 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.160177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770372f-eec5-424a-9046-a81b265df1f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:52.160488 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770372f-eec5-424a-9046-a81b265df1f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.161878 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770372f-eec5-424a-9046-a81b265df1f2 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.173427 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 288/556] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:52.174725 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9072020-a67d-4005-ab57-698ee718c71d None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.175045 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9072020-a67d-4005-ab57-698ee718c71d None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.175322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9072020-a67d-4005-ab57-698ee718c71d None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.177071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 289/557] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:52.179111 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9072020-a67d-4005-ab57-698ee718c71d None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.220783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3da5f-c342-4df0-b955-5e1f51606155 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.259469 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c55e74-cb90-4225-867b-a7822924fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.260587 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55e74-cb90-4225-867b-a7822924fd61 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.261082 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55e74-cb90-4225-867b-a7822924fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.261543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c55e74-cb90-4225-867b-a7822924fd61 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.262527 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c55e74-cb90-4225-867b-a7822924fd61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:52.263049 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c55e74-cb90-4225-867b-a7822924fd61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.264674 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c55e74-cb90-4225-867b-a7822924fd61 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.278085 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 269/558] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:52.279654 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47aafd-6334-49f3-854c-d985ceead44d None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.280201 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47aafd-6334-49f3-854c-d985ceead44d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.280711 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47aafd-6334-49f3-854c-d985ceead44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.289966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9072020-a67d-4005-ab57-698ee718c71d None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.298857 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-da47aafd-6334-49f3-854c-d985ceead44d None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:52.300390 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-da47aafd-6334-49f3-854c-d985ceead44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:52.324365 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.324933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.325247 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.325525 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.332896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.334413 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:52.348839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:52.350065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:52.351081 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 290/559] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:52.351607 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.383572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 270/560] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:52.384921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47aafd-6334-49f3-854c-d985ceead44d None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.469925 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa0c41-6a6e-4aa8-a312-c5ea26d1592b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.487373 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47aafd-6334-49f3-854c-d985ceead44d None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.506599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af053f-07f3-433e-9574-4dba14770300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.507403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af053f-07f3-433e-9574-4dba14770300 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.507688 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af053f-07f3-433e-9574-4dba14770300 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.507967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af053f-07f3-433e-9574-4dba14770300 None admin] PATH_INFO: `/v3/users/swift` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.515768 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af053f-07f3-433e-9574-4dba14770300 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.517258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af053f-07f3-433e-9574-4dba14770300 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.524966 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.525673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.525962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.526244 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.527071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:52.527386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.528825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-03af053f-07f3-433e-9574-4dba14770300 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:52.533317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:52.534341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:28:52.535348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:52.539387 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 291/561] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:28:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:52.540391 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f7f2-84bd-4c4f-91d1-00caf1a671e1 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.540685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f7f2-84bd-4c4f-91d1-00caf1a671e1 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.540966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f7f2-84bd-4c4f-91d1-00caf1a671e1 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.546584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 271/562] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:52.546736 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 292/563] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:52.546736 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7f7f2-84bd-4c4f-91d1-00caf1a671e1 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.546881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.644353 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201064d-ef6d-456b-8a48-fc17defaa2b7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.645250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7f7f2-84bd-4c4f-91d1-00caf1a671e1 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.676234 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5fff5-747b-4b9a-83b7-8f6d9561adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.677061 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5fff5-747b-4b9a-83b7-8f6d9561adcd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.677350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5fff5-747b-4b9a-83b7-8f6d9561adcd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.677631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5fff5-747b-4b9a-83b7-8f6d9561adcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.678125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.678789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.679071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.679348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.680268 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:52.680581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.682149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.687502 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5fff5-747b-4b9a-83b7-8f6d9561adcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-1ad5fff5-747b-4b9a-83b7-8f6d9561adcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:52.695788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:52.697417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:52.698301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 272/564] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:52.699056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f20e55-e057-45bd-a7f7-d787cfbc36a9 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.699056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f20e55-e057-45bd-a7f7-d787cfbc36a9 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.699201 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f20e55-e057-45bd-a7f7-d787cfbc36a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.704378 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 293/565] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:28:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:52.705676 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.716270 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-35f20e55-e057-45bd-a7f7-d787cfbc36a9 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:52.717522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-35f20e55-e057-45bd-a7f7-d787cfbc36a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:52.780717 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 273/566] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:52.782036 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87686f-c360-4980-a771-fad8c7a11a10 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.782327 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87686f-c360-4980-a771-fad8c7a11a10 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.782605 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87686f-c360-4980-a771-fad8c7a11a10 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.783845 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 274/567] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:52.784554 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87686f-c360-4980-a771-fad8c7a11a10 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.816746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72533c4f-015d-4906-bafe-bcf04fde2036 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.848343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb2143-f130-42c4-85dd-db9ac34904b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.849256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2143-f130-42c4-85dd-db9ac34904b6 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.849545 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2143-f130-42c4-85dd-db9ac34904b6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.849819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb2143-f130-42c4-85dd-db9ac34904b6 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/17730dd371184663b2c0eb091cc668a3/roles/582e6b7be6a445019c4cd90d2dcd594a` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.864196 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb2143-f130-42c4-85dd-db9ac34904b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=17730dd371184663b2c0eb091cc668a3, role_id=582e6b7be6a445019c4cd90d2dcd594a)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.865669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb2143-f130-42c4-85dd-db9ac34904b6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.879237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 294/568] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:52 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/17730dd371184663b2c0eb091cc668a3/roles/582e6b7be6a445019c4cd90d2dcd594a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:52.880026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb2143-f130-42c4-85dd-db9ac34904b6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:52.886905 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87686f-c360-4980-a771-fad8c7a11a10 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:52.926148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c1925-113c-48fa-b694-5ad4459bd4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:52.926843 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1925-113c-48fa-b694-5ad4459bd4a4 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.927135 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1925-113c-48fa-b694-5ad4459bd4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.927408 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1925-113c-48fa-b694-5ad4459bd4a4 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.928174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c1925-113c-48fa-b694-5ad4459bd4a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:52.928488 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c1925-113c-48fa-b694-5ad4459bd4a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:52.929936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c1925-113c-48fa-b694-5ad4459bd4a4 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:52.939432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 275/569] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:52.941214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bc974-ba1b-46d0-8221-3b636c2b1380 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:52.941514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bc974-ba1b-46d0-8221-3b636c2b1380 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:52.941794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bc974-ba1b-46d0-8221-3b636c2b1380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:52.959074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-8a2bc974-ba1b-46d0-8221-3b636c2b1380 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:52.960313 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-8a2bc974-ba1b-46d0-8221-3b636c2b1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:52.991445 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb2143-f130-42c4-85dd-db9ac34904b6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.026964 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810401a-a5d6-4da1-a8f4-f49ce2d8f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.027676 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810401a-a5d6-4da1-a8f4-f49ce2d8f58c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.027911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810401a-a5d6-4da1-a8f4-f49ce2d8f58c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.028189 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810401a-a5d6-4da1-a8f4-f49ce2d8f58c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.037322 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 276/570] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:53.037322 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bc974-ba1b-46d0-8221-3b636c2b1380 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.113675 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810401a-a5d6-4da1-a8f4-f49ce2d8f58c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:53.115458 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810401a-a5d6-4da1-a8f4-f49ce2d8f58c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:53.146093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bc974-ba1b-46d0-8221-3b636c2b1380 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.182612 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ce24c-de81-42c1-be54-778ddd4ecc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.183305 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce24c-de81-42c1-be54-778ddd4ecc07 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.183591 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce24c-de81-42c1-be54-778ddd4ecc07 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.183868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ce24c-de81-42c1-be54-778ddd4ecc07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.203009 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ce24c-de81-42c1-be54-778ddd4ecc07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-349ce24c-de81-42c1-be54-778ddd4ecc07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:53.210859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:53.211958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 277/571] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:53.218040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ce24c-de81-42c1-be54-778ddd4ecc07 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.218834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 295/572] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:28:53.218834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810401a-a5d6-4da1-a8f4-f49ce2d8f58c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.314529 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ce24c-de81-42c1-be54-778ddd4ecc07 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.315531 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810401a-a5d6-4da1-a8f4-f49ce2d8f58c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.346153 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.347470 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.347756 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.348024 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.348602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe32a5-7166-40a1-ae4e-781dee7402dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.349424 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe32a5-7166-40a1-ae4e-781dee7402dd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.349707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe32a5-7166-40a1-ae4e-781dee7402dd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.349983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe32a5-7166-40a1-ae4e-781dee7402dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.359361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:53.360254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe32a5-7166-40a1-ae4e-781dee7402dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:53.360808 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-46fe32a5-7166-40a1-ae4e-781dee7402dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:53.368738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:53.369915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 296/573] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:53.370816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe32a5-7166-40a1-ae4e-781dee7402dd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:53.382430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:53.383576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:53.384748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:53.384748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:53.384748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:53.384748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:53.390829 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 278/574] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:53.390829 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.468653 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe32a5-7166-40a1-ae4e-781dee7402dd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.490120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb87e8-feb0-456d-99e9-0d57cddd9f94 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.501979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.502862 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.503217 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.503570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.513981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:53.515536 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:53.523228 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.523970 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.524324 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.524669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.525535 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:53.525925 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:53.527337 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:53.533097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:53.534228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:53.540355 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:53.540355 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:53.540355 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:53.540355 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:53.540355 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 297/575] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:53.540355 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.547498 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 279/576] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:53.547498 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.640734 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700624-9ce1-4098-84f0-25cc45575240 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.656878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb923e3d-e2ae-4885-99e6-94eddbfb96b6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.680777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.681490 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.681779 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.682062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.682833 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:53.683144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:53.684530 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:53.690173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.690868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.691147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.691427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.692156 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:53.692464 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:53.698472 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 298/577] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:53.698655 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:53.698771 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.707424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 280/578] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:53.708922 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.797916 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b6137-8a19-4040-b02c-1cf8675f93b7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.814106 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdecb2-cf20-4844-929a-fff6e70a40dd None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.830289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.831622 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.832018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.832400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.833290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:53.833709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:53.835182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:53.851090 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 299/579] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:53.851090 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.851918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a7124-1e11-441a-9567-80722880825c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:53.852786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a7124-1e11-441a-9567-80722880825c None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:53.853167 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a7124-1e11-441a-9567-80722880825c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:53.853530 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a7124-1e11-441a-9567-80722880825c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:53.863861 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a7124-1e11-441a-9567-80722880825c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:53.865416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a7124-1e11-441a-9567-80722880825c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-db6a7124-1e11-441a-9567-80722880825c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:53.883219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:53.884311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:53.889744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:53.889744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:53.889744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:53.889744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:53.889744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 281/580] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:53.889744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a7124-1e11-441a-9567-80722880825c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:53.967054 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70bf7c-8408-4d0a-907e-8fdb49887f21 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:53.992795 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a7124-1e11-441a-9567-80722880825c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.007544 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.008618 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.008905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.009211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.021373 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.022845 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.028203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.029002 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.029343 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.029618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] PATH_INFO: `/v3/users/glance` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.036788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.038274 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:54.041143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:54.042061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:54.047985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:54.047985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:54.047985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:54.047985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:54.047985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 300/581] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:54.047985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:54.056760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:54.058103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:54.059277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 282/582] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:54.062298 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.154272 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62bca4-e829-494e-9cb4-386868c7f8e7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.170325 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa12ac6-cef7-44ca-bd9c-85d010507344 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.193757 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.194925 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.195390 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.195820 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.202674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.204302 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.206737 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.207435 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.207697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.207981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.208723 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:54.209060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.210462 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:54.219822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:54.220950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 301/583] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:54.222171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.224598 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 283/584] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:54.225738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.328064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d13ff-5820-4e6a-b0d0-e3b93a2958df None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.333912 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeac38a-33fe-478b-9b35-4cd3c71073fa None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.361210 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.361883 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.362172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.363047 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.363814 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:54.364128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.365590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.366904 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.367572 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.367860 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.368147 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.368878 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:54.369221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.370591 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.391026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 302/585] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:54.391026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.391298 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 284/586] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:54.391298 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.482564 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffc734-d87e-4240-a63f-c9cea66efdbe None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.488772 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a02f53-a98f-4a27-bc83-0bec25dca504 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.515644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.516439 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.516725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.517007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.522319 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.522982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.523282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.523578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.524218 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.524331 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:54.524617 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.525704 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.526096 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.537638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 303/587] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:54.539293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:54.540038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:54.541074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:28:54.542664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:54.545362 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 285/588] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:54.546274 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.644410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19c6b7-6697-4408-b8a4-03b023b8ed3e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.650473 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e5588-9562-460c-bd05-aa8e8a081454 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.678736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b990-966b-411a-af23-f5b97986243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.679559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b990-966b-411a-af23-f5b97986243b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.679841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b990-966b-411a-af23-f5b97986243b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.680125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608b990-966b-411a-af23-f5b97986243b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.684607 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.685448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.685730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.686006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.687811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608b990-966b-411a-af23-f5b97986243b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.689297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608b990-966b-411a-af23-f5b97986243b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.696754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.698292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-0608b990-966b-411a-af23-f5b97986243b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:54.704193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:54.705214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:54.706258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 286/589] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:54.707479 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b990-966b-411a-af23-f5b97986243b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:54.720764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:54.722309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:54.723266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:54.723266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:54.723266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:54.723266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:54.723266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 304/590] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:54.723560 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.810387 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608b990-966b-411a-af23-f5b97986243b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.822266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dee4f-0dcc-4016-a042-6716e9c232f0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:54.843361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.844103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.844454 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.844796 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.845740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=a752b7105b754628a153e396a9110653) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:54.846124 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.848094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.862148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 287/591] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/role_assignments?user.id=a752b7105b754628a153e396a9110653&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:54.862148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.867879 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:54.868690 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:54.869062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:54.869443 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:54.870406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:54.870801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:54.873029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:54.897740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 305/592] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:54.898816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:54.975800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4ffe6-6267-48e3-9ae9-48925c3333d9 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.012944 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.013784 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.014146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.014497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.015473 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:55.015867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.017539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.030578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634fc36-503f-4be2-91fe-080348b2fe2c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.040250 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 288/593] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:55.041348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.089571 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.090350 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.090705 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.091053 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.092259 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:55.092698 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.094856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.117941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 306/594] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:55.118895 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.168772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150cbcf-0247-4de5-8762-ad3a4b0d6454 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.215107 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.215917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.216201 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.216479 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.226850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.228480 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.240023 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3f69a-ddfb-43da-92c9-ca38d254c899 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:55.246770 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:55.247732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:55.248694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:55.248694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:55.248694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:55.248694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:55.248694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 289/595] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:55.249423 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.274507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6429ab-f087-4a1b-92db-701ed2e9fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.275309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6429ab-f087-4a1b-92db-701ed2e9fa56 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.275591 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6429ab-f087-4a1b-92db-701ed2e9fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.275870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6429ab-f087-4a1b-92db-701ed2e9fa56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.286420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6429ab-f087-4a1b-92db-701ed2e9fa56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.287877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6429ab-f087-4a1b-92db-701ed2e9fa56 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-9e6429ab-f087-4a1b-92db-701ed2e9fa56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:55.305601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:55.306556 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:55.307590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:55.307590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:55.307590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:55.307590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:55.307590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 307/596] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:55.308429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342fb7-c597-470a-a899-f65aabfa6b97 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.308429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342fb7-c597-470a-a899-f65aabfa6b97 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.308687 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1342fb7-c597-470a-a899-f65aabfa6b97 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.310149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 308/597] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:55.310780 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1342fb7-c597-470a-a899-f65aabfa6b97 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.354999 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d011f94-eda0-4860-a00d-03ab63a7aefe None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.391632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.392468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.392757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.393469 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.400307 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.401803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.411854 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1342fb7-c597-470a-a899-f65aabfa6b97 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:55.416668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:55.417881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:55.419076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:55.419076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:55.419076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:55.419076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:55.419076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:55.419076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:55.419076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:55.419076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:55.425890 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 290/598] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:55.426893 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.447800 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ebb21-adbc-47d6-a185-24de9836d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.448526 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ebb21-adbc-47d6-a185-24de9836d7fb None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.448809 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ebb21-adbc-47d6-a185-24de9836d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.449120 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ebb21-adbc-47d6-a185-24de9836d7fb None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.449849 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ebb21-adbc-47d6-a185-24de9836d7fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:55.450154 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ebb21-adbc-47d6-a185-24de9836d7fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.451525 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ebb21-adbc-47d6-a185-24de9836d7fb None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.474321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 309/599] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:55.474321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714eb51-a878-49a8-beca-fec9f5b3de92 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.474321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714eb51-a878-49a8-beca-fec9f5b3de92 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.474321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714eb51-a878-49a8-beca-fec9f5b3de92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.490537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-2714eb51-a878-49a8-beca-fec9f5b3de92 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:55.491735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-2714eb51-a878-49a8-beca-fec9f5b3de92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:55.536627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c703380-dbf9-4b88-8705-fb75444b37dd None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.557084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 310/600] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:55.557084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714eb51-a878-49a8-beca-fec9f5b3de92 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.581938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.582763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.583056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.583655 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.584670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:55.584985 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.587519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.601471 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 291/601] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:55.602372 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.661961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714eb51-a878-49a8-beca-fec9f5b3de92 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.697449 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.697946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.698500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.698622 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.699415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:55.699700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.701183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.701307 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f0d3f-40e2-45f2-9cdd-c738d283b9aa None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.714233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 311/602] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:55.715400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.739644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.740473 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.740764 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.741059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.747663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.749161 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:55.762638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:55.763701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 292/603] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:55.764739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.826794 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4de055-f08a-4e03-a334-14f8fb277c3c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.862350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cefe62-ab47-420e-af03-adc0a17b8b4c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:55.862935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938111b-53d7-4696-88b8-20c4c82e8ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.863736 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938111b-53d7-4696-88b8-20c4c82e8ef4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.864020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938111b-53d7-4696-88b8-20c4c82e8ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.864296 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2938111b-53d7-4696-88b8-20c4c82e8ef4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.874213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2938111b-53d7-4696-88b8-20c4c82e8ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-2938111b-53d7-4696-88b8-20c4c82e8ef4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:55.881950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:55.883062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 312/604] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:55.884074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938111b-53d7-4696-88b8-20c4c82e8ef4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.901145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:55.902031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:55.902320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:55.902602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:55.909935 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:55.911438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:55.926007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:55.926948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:55.927932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 293/605] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:55.928454 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:55.986419 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2938111b-53d7-4696-88b8-20c4c82e8ef4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.021468 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec70ec-ad98-46c6-8923-9681b90f9af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.022363 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec70ec-ad98-46c6-8923-9681b90f9af4 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.022651 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec70ec-ad98-46c6-8923-9681b90f9af4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.022930 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec70ec-ad98-46c6-8923-9681b90f9af4 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/4e125e49ac664c749fe6eede9531920d/roles/582e6b7be6a445019c4cd90d2dcd594a` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.029381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1fca4-4de7-4b5f-b81a-2b38a49971ab None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.038078 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec70ec-ad98-46c6-8923-9681b90f9af4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=4e125e49ac664c749fe6eede9531920d, role_id=582e6b7be6a445019c4cd90d2dcd594a)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.039579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec70ec-ad98-46c6-8923-9681b90f9af4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.061174 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 313/606] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:55 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/4e125e49ac664c749fe6eede9531920d/roles/582e6b7be6a445019c4cd90d2dcd594a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:56.063128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec70ec-ad98-46c6-8923-9681b90f9af4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:56.067007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.067728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.068016 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.068297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.069161 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:56.069478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.072176 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.099395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 294/607] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:56.100419 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:56.211517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec70ec-ad98-46c6-8923-9681b90f9af4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.216993 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ef69d-fd27-48d9-bd54-6de8d622ed13 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.252650 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.255211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.255740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.256335 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.256937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.257639 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.257764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] RBAC: Adding query filter params (name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:56.257940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.258196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.258694 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.258999 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:56.259314 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.260747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.260869 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.274479 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 295/609] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:56.274860 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 314/609] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:56.275425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:56.278738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:56.382152 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19ab61-0b84-4348-b2d6-b19a05448efd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.383961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a00004-d576-4c3c-906d-eac757ba9d8c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.417613 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.418245 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc243b73-bdeb-40cc-8c83-2ea96088b897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.419084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.419222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc243b73-bdeb-40cc-8c83-2ea96088b897 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.419402 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc243b73-bdeb-40cc-8c83-2ea96088b897 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.419690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc243b73-bdeb-40cc-8c83-2ea96088b897 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/365477e05c1f48beac7f88e6a0707bd6/roles/582e6b7be6a445019c4cd90d2dcd594a` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.420149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.420730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.432308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.434130 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.435914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc243b73-bdeb-40cc-8c83-2ea96088b897 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=365477e05c1f48beac7f88e6a0707bd6, role_id=582e6b7be6a445019c4cd90d2dcd594a)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.437532 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc243b73-bdeb-40cc-8c83-2ea96088b897 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.452972 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 296/610] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:56 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/365477e05c1f48beac7f88e6a0707bd6/roles/582e6b7be6a445019c4cd90d2dcd594a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:56.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:56.459000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:56.459941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:56.459941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:56.459941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:56.459941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:56.460208 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc243b73-bdeb-40cc-8c83-2ea96088b897 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:56.467196 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 315/611] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:56.469924 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:56.611351 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181797d-f7c4-4015-89ea-2dd46b2c183c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.673777 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc243b73-bdeb-40cc-8c83-2ea96088b897 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.685807 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.686503 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.686789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.687063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.687969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:56.688174 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.689576 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.710590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 316/612] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:56.715887 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b5405-a88f-4292-999a-c68db3af3760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.716107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:56.716794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5405-a88f-4292-999a-c68db3af3760 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.717106 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5405-a88f-4292-999a-c68db3af3760 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.717393 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b5405-a88f-4292-999a-c68db3af3760 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/5382d7219c244430ae6f37d6b32e7d8a/roles/582e6b7be6a445019c4cd90d2dcd594a` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.734991 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b5405-a88f-4292-999a-c68db3af3760 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=5382d7219c244430ae6f37d6b32e7d8a, role_id=582e6b7be6a445019c4cd90d2dcd594a)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.736765 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b5405-a88f-4292-999a-c68db3af3760 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.759705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 297/613] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:56 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/5382d7219c244430ae6f37d6b32e7d8a/roles/582e6b7be6a445019c4cd90d2dcd594a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:28:56.843448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330be1f1-e92e-4abc-b138-e8d5be239764 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:56.882663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:56.883422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:56.883754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:56.884038 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] PATH_INFO: `/v3/users/nova` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:56.892253 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:56.893786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:56.910210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:56.911301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:56.912403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 317/614] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:28:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:56.919793 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b5405-a88f-4292-999a-c68db3af3760 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:57.062138 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b5405-a88f-4292-999a-c68db3af3760 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:57.103691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f41818-a4a0-4118-b044-93497350ccdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:57.103691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41818-a4a0-4118-b044-93497350ccdf None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:57.103691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41818-a4a0-4118-b044-93497350ccdf None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:57.103691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f41818-a4a0-4118-b044-93497350ccdf None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:57.104073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f41818-a4a0-4118-b044-93497350ccdf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:57.104285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f41818-a4a0-4118-b044-93497350ccdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:57.105686 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f41818-a4a0-4118-b044-93497350ccdf None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:57.127021 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 298/615] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:28:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:57.135499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:57.221646 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9e003-357a-4bef-9ea3-276cd2a8ac8c None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:57.222620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9e003-357a-4bef-9ea3-276cd2a8ac8c None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:57.223008 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9e003-357a-4bef-9ea3-276cd2a8ac8c None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:57.224816 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 299/616] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:28:57.229720 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b408-5a66-4eee-a787-cea376d6cee6 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:57.230016 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b408-5a66-4eee-a787-cea376d6cee6 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:57.230298 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b408-5a66-4eee-a787-cea376d6cee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:57.247212 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-bf44b408-5a66-4eee-a787-cea376d6cee6 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:57.247418 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8051803-d3e6-4953-8ce2-4f592dee8c7d None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:57.248410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-bf44b408-5a66-4eee-a787-cea376d6cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:57.287798 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c052369-42a6-4fbc-9a95-c01cad20137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:57.288464 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c052369-42a6-4fbc-9a95-c01cad20137b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:57.288746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c052369-42a6-4fbc-9a95-c01cad20137b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:57.289023 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c052369-42a6-4fbc-9a95-c01cad20137b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:57.353867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 300/617] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:57.402823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c052369-42a6-4fbc-9a95-c01cad20137b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:57.404327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c052369-42a6-4fbc-9a95-c01cad20137b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:57.421526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44b408-5a66-4eee-a787-cea376d6cee6 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:57.558791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44b408-5a66-4eee-a787-cea376d6cee6 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:57.563157 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 318/618] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:28:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:28:57.574110 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c052369-42a6-4fbc-9a95-c01cad20137b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:57.606346 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83debc7-0464-41f0-ae0e-b0b25a1f1120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:57.607432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83debc7-0464-41f0-ae0e-b0b25a1f1120 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:57.607713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83debc7-0464-41f0-ae0e-b0b25a1f1120 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:57.607988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83debc7-0464-41f0-ae0e-b0b25a1f1120 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:57.630663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83debc7-0464-41f0-ae0e-b0b25a1f1120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-e83debc7-0464-41f0-ae0e-b0b25a1f1120 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:57.642460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:57.644122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:57.645017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 301/619] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:57.652175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83debc7-0464-41f0-ae0e-b0b25a1f1120 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:57.677400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c052369-42a6-4fbc-9a95-c01cad20137b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:57.713797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:57.714751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:57.715037 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:57.715314 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:57.726831 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:57.728298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:57.747691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:57.749642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:57.751172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:57.751172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:57.751172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:57.751172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:57.751172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 319/620] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:57.760437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:57.818843 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83debc7-0464-41f0-ae0e-b0b25a1f1120 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:57.858893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3fb90-da39-400e-ba1e-612f6fcd6b81 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:57.866735 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:57.867424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:57.867713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:57.867992 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:57.868761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:57.869094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:57.870513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:57.888922 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 302/621] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:57.894550 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:57.895220 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:57.895485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:57.895683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:57.895855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:57.896647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:57.896958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:57.898397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:57.910748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 320/622] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:57.916309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:58.027390 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb244d4-98a8-475d-9889-15e28a955ad3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:58.064502 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:58.066054 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.066436 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.066840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.074074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:58.075631 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:58.090082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:58.091262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:58.092522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:58.092522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:58.092522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:58.092522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:58.092522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:58.092522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:58.092522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:58.092522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:58.094252 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 321/623] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:58.102346 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:58.142291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd7858-98a8-4a52-b565-fdfc01c8529d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:58.193031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:58.194703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.196035 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.196529 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.217545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:58.219430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:58.255251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:58.256637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:58.257838 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:58.257838 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:58.257838 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:58.257838 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:58.261426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 303/624] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:58.270021 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:58.294347 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f33222-cc53-4b32-8c2e-219d3eed245e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:58.348394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894662f-377b-436d-b651-3566ad50021a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:58.349082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894662f-377b-436d-b651-3566ad50021a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.349366 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894662f-377b-436d-b651-3566ad50021a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.349638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894662f-377b-436d-b651-3566ad50021a None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.350440 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894662f-377b-436d-b651-3566ad50021a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:58.350752 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894662f-377b-436d-b651-3566ad50021a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:58.352197 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894662f-377b-436d-b651-3566ad50021a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:58.369617 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 322/625] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:58.375127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894662f-377b-436d-b651-3566ad50021a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:58.436828 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035a8a-4c83-48ed-b98f-42c5f38549ec None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:58.475193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:58.475871 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.476159 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.476440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.477208 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:58.477530 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:58.478914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:58.492890 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 304/626] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:58.502203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:58.540573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894662f-377b-436d-b651-3566ad50021a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:58.583774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:58.584370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.584654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.584931 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.585826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=17730dd371184663b2c0eb091cc668a3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:58.586145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:58.587631 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:58.603000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 323/627] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:28:58 2022] GET /identity/v3/role_assignments?user.id=17730dd371184663b2c0eb091cc668a3&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:58.673460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64760d36-5259-4c9e-bd90-b1eb117615aa None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:58.720420 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4679b7a-16ef-4ed5-916c-1eb8204ef2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:58.721262 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4679b7a-16ef-4ed5-916c-1eb8204ef2da None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.721600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4679b7a-16ef-4ed5-916c-1eb8204ef2da None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.722382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4679b7a-16ef-4ed5-916c-1eb8204ef2da None admin] PATH_INFO: `/v3/users/swift` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.730200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4679b7a-16ef-4ed5-916c-1eb8204ef2da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:58.731651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4679b7a-16ef-4ed5-916c-1eb8204ef2da None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-c4679b7a-16ef-4ed5-916c-1eb8204ef2da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:28:58.748976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:58.751410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:58.753881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 305/628] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:28:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:58.757175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:58.813999 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d2c3-eda1-4386-9836-f8641ea3974b None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.813999 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d2c3-eda1-4386-9836-f8641ea3974b None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.813999 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d2c3-eda1-4386-9836-f8641ea3974b None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.814663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 306/629] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:58.820104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fac8a-07ae-48c6-8ae8-2c6ebe64c2e3 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.821005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fac8a-07ae-48c6-8ae8-2c6ebe64c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.821535 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fac8a-07ae-48c6-8ae8-2c6ebe64c2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.839550 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-5f3fac8a-07ae-48c6-8ae8-2c6ebe64c2e3 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:58.841094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-5f3fac8a-07ae-48c6-8ae8-2c6ebe64c2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:58.870985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e216f-9409-4f70-96b1-df7ceb4dbc2b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:58.917513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 307/630] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:28:58.919205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:58.919913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:58.920201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:58.920487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:58.921388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:58.921707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:58.923186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:58.938419 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fac8a-07ae-48c6-8ae8-2c6ebe64c2e3 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:58.949833 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 324/631] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:28:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:58.955497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.041175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fac8a-07ae-48c6-8ae8-2c6ebe64c2e3 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.056786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a6ba7-4e7d-4945-9188-100e0a91f3d5 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.078164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97890c8a-db98-40d5-a5f3-247e053ed2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.079924 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97890c8a-db98-40d5-a5f3-247e053ed2ad None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.080474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97890c8a-db98-40d5-a5f3-247e053ed2ad None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.081880 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97890c8a-db98-40d5-a5f3-247e053ed2ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.095422 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97890c8a-db98-40d5-a5f3-247e053ed2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-97890c8a-db98-40d5-a5f3-247e053ed2ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:59.105906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:59.107259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:59.108150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:59.110708 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 308/632] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:59.114783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.114783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.114783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.114783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.117750 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97890c8a-db98-40d5-a5f3-247e053ed2ad None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.124918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:59.126527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:59.149127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:59.150348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:59.151499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:59.151499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:59.151499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:59.151499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:59.157239 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 325/633] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:28:59.158640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.238763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97890c8a-db98-40d5-a5f3-247e053ed2ad None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.274876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea87705-eb8e-4da0-8963-4a031ffb3db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.275453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f423b-2be5-4b4d-aedc-07131a4682c9 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.275834 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea87705-eb8e-4da0-8963-4a031ffb3db0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.276232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea87705-eb8e-4da0-8963-4a031ffb3db0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.276660 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea87705-eb8e-4da0-8963-4a031ffb3db0 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.278225 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea87705-eb8e-4da0-8963-4a031ffb3db0 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:59.279483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea87705-eb8e-4da0-8963-4a031ffb3db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:59.281194 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea87705-eb8e-4da0-8963-4a031ffb3db0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:59.293492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 309/634] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:59.294925 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d536dd1-650a-46f0-bc05-8d7549302af4 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.295226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d536dd1-650a-46f0-bc05-8d7549302af4 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.295511 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d536dd1-650a-46f0-bc05-8d7549302af4 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.296771 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 310/635] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:59.301777 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d536dd1-650a-46f0-bc05-8d7549302af4 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.319144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae922bde-be3f-4a8b-a741-78ea39039d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.319996 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae922bde-be3f-4a8b-a741-78ea39039d43 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.320392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae922bde-be3f-4a8b-a741-78ea39039d43 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.320789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae922bde-be3f-4a8b-a741-78ea39039d43 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.321704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae922bde-be3f-4a8b-a741-78ea39039d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:59.322131 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae922bde-be3f-4a8b-a741-78ea39039d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:59.323625 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae922bde-be3f-4a8b-a741-78ea39039d43 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:59.338163 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 326/636] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:28:59.339677 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed58a65-7958-42ad-b59f-4fdedd52e1d4 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.340116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed58a65-7958-42ad-b59f-4fdedd52e1d4 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.340538 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed58a65-7958-42ad-b59f-4fdedd52e1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.357898 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-eed58a65-7958-42ad-b59f-4fdedd52e1d4 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:59.359392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-eed58a65-7958-42ad-b59f-4fdedd52e1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:59.442408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 327/637] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:59.449525 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed58a65-7958-42ad-b59f-4fdedd52e1d4 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.453992 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d536dd1-650a-46f0-bc05-8d7549302af4 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.495638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.496887 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.497325 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.497898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.511678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:59.513682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:28:59.537347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:59.538599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:59.539759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:28:59.539759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:28:59.539759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:28:59.539759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:59.541970 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 311/638] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:59.542998 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.597249 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed58a65-7958-42ad-b59f-4fdedd52e1d4 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.634030 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.634750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.635031 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.635300 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.641974 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:59.643488 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:59.653995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39234b3a-6e00-4b27-b599-06c8f6c2ff12 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:28:59.660236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:59.661237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:28:59.662309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 328/639] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:59.664231 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.691654 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ec6af-e000-4482-a03d-bd0817f479e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.692456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ec6af-e000-4482-a03d-bd0817f479e0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.692741 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ec6af-e000-4482-a03d-bd0817f479e0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.693086 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ec6af-e000-4482-a03d-bd0817f479e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.704413 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ec6af-e000-4482-a03d-bd0817f479e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5f0ec6af-e000-4482-a03d-bd0817f479e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:28:59.712263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:28:59.713286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:28:59.714220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 312/640] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:28:59.716450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a21106-d789-4371-a237-c93ce489065e None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.716450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a21106-d789-4371-a237-c93ce489065e None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.716450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a21106-d789-4371-a237-c93ce489065e None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.721261 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 313/641] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:28:59.721261 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a21106-d789-4371-a237-c93ce489065e None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.772840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01bb63-b5e2-40df-ba85-babeafea0e5f None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.805804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.806475 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.806763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.807040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.807776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:59.808087 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:59.809530 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:59.821827 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 329/642] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:59.826961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.831038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a21106-d789-4371-a237-c93ce489065e None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.864288 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde39bd-4cae-4bd3-a8f8-c51ee8d95bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.864958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde39bd-4cae-4bd3-a8f8-c51ee8d95bf3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.865273 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde39bd-4cae-4bd3-a8f8-c51ee8d95bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.865552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde39bd-4cae-4bd3-a8f8-c51ee8d95bf3 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.866364 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde39bd-4cae-4bd3-a8f8-c51ee8d95bf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:59.866669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde39bd-4cae-4bd3-a8f8-c51ee8d95bf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:59.868092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde39bd-4cae-4bd3-a8f8-c51ee8d95bf3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:59.886310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 314/643] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:59.886310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165d6a0-83f7-430f-8998-ac318fd98911 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.886310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165d6a0-83f7-430f-8998-ac318fd98911 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.886310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165d6a0-83f7-430f-8998-ac318fd98911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.902937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-e165d6a0-83f7-430f-8998-ac318fd98911 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:28:59.904207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-e165d6a0-83f7-430f-8998-ac318fd98911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:28:59.925662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b37fa-fa9e-4632-9747-23f44c1a1f46 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:28:59.960644 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:28:59.961410 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:28:59.961712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:28:59.962045 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:28:59.962809 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] RBAC: Adding query filter params (name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:28:59.963121 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:28:59.964581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:28:59.969472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 315/644] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:28:59.970316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165d6a0-83f7-430f-8998-ac318fd98911 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:28:59.982456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 330/645] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:28:59.983825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.073472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165d6a0-83f7-430f-8998-ac318fd98911 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.087680 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0618b-11ee-4749-bb9e-d688a565ff96 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.107891 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.108767 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.109163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.109513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] PATH_INFO: `/v3/users/glance` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.118776 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.120768 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:00.125067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09465595-b804-48b9-b8b1-40dd08770a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.125953 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09465595-b804-48b9-b8b1-40dd08770a16 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.126239 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09465595-b804-48b9-b8b1-40dd08770a16 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.126514 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09465595-b804-48b9-b8b1-40dd08770a16 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/a752b7105b754628a153e396a9110653/roles/582e6b7be6a445019c4cd90d2dcd594a` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:00.138757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:00.139876 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:29:00.141163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:00.141867 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09465595-b804-48b9-b8b1-40dd08770a16 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=a752b7105b754628a153e396a9110653, role_id=582e6b7be6a445019c4cd90d2dcd594a)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.147114 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 316/646] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:28:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:00.147519 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09465595-b804-48b9-b8b1-40dd08770a16 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:00.147637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.157012 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 331/647] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:28:59 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/a752b7105b754628a153e396a9110653/roles/582e6b7be6a445019c4cd90d2dcd594a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:29:00.158002 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09465595-b804-48b9-b8b1-40dd08770a16 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.279434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cae05-1a28-4258-a76a-fffebee83d1d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.288459 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09465595-b804-48b9-b8b1-40dd08770a16 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.318841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.320019 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.320513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.320988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.329140 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6094a6a-245b-4eaa-aac7-3fb96799e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.330118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6094a6a-245b-4eaa-aac7-3fb96799e256 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.330479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6094a6a-245b-4eaa-aac7-3fb96799e256 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.331322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6094a6a-245b-4eaa-aac7-3fb96799e256 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.333275 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.335008 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:00.345226 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6094a6a-245b-4eaa-aac7-3fb96799e256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-d6094a6a-245b-4eaa-aac7-3fb96799e256 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:00.353485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:00.354679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:00.355802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:00.361103 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 332/648] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:00.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:00.362864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:00.364078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:00.364078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:00.364078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:00.364078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:00.364366 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6094a6a-245b-4eaa-aac7-3fb96799e256 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.364462 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 317/649] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:00.370027 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.472990 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6094a6a-245b-4eaa-aac7-3fb96799e256 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.479824 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fea79-c18f-4f65-8b98-e0645919da0a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.509169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.510272 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.511021 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.511827 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.512890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:00.513390 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.514969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:00.515937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.516706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.517121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.517502 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.518887 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:00.518887 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.520210 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:00.536220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 318/650] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:00.536220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.541674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 333/651] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:00.542748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.654672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803ca7e-c0fd-4a9a-8ac8-76499d5ed218 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.658289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe22475-e5c2-4895-9951-040c8c794457 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.691906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.692688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.693264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.693480 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.694338 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:00.694754 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.696241 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:00.703416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.704553 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.705005 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.705567 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.706097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 319/652] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:00.707014 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.715106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.716852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:00.738114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:00.739326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:29:00.740554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:00.747089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 334/653] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:00.747089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.843885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df9cef-73d4-485f-87f0-802567112cfa None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.849850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ff68-bab2-4f6f-86c2-b5aacc1b7cf1 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:00.885239 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.886201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.886583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.886963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.888386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb0f04-ab00-4361-bc37-35179184a9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:00.889438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb0f04-ab00-4361-bc37-35179184a9fb None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:00.889898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb0f04-ab00-4361-bc37-35179184a9fb None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:00.890645 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb0f04-ab00-4361-bc37-35179184a9fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:00.898762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.900348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:00.919665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:00.920863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:00.922073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:00.922073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:00.922073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:00.922073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:00.923635 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 335/654] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:00.924550 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:00.977263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb0f04-ab00-4361-bc37-35179184a9fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:00.978895 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb0f04-ab00-4361-bc37-35179184a9fb None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.034062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85c7a1-6ae9-42cd-a735-d7c92c566254 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.069167 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.070194 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.070708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.071206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.072301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:01.072848 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.074533 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.084903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 320/655] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:29:01.084903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb0f04-ab00-4361-bc37-35179184a9fb None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.104395 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 336/656] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:29:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:01.105597 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.199790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb0f04-ab00-4361-bc37-35179184a9fb None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.211685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab26ed0-c961-4ffb-915f-7d010a84a9d0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.237501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.238082 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.238369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.238654 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.239395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:01.239702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.241095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.250958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.252074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.252578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.252732 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 321/657] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:01.253287 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.253566 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.266859 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.269771 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:01.294064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:01.295063 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:01.296006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:01.296006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:01.296006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:01.296006 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:01.303543 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 337/658] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:01.303543 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.379223 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513198d-bfe2-48a3-92f3-63cedf95ae9a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.406033 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5120e-5989-4c9e-9d2b-edc03d11cb0d None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.415537 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246c6a5-daae-4296-b820-92bcfaeb90a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.416201 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c6a5-daae-4296-b820-92bcfaeb90a6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.416496 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c6a5-daae-4296-b820-92bcfaeb90a6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.416826 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c6a5-daae-4296-b820-92bcfaeb90a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.439041 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.439844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.440129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.440936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.447977 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.449458 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:01.464998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:01.466089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:01.467122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 338/659] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:01.467764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.493507 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246c6a5-daae-4296-b820-92bcfaeb90a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.495171 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246c6a5-daae-4296-b820-92bcfaeb90a6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.566930 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f282f7-b70b-4482-8a60-701f609d0943 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.601196 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.602018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.602309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.602586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.607665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 322/660] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:29:01.607807 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:01.607807 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623460-1652-46d4-983a-e56b2cb03f75 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.608008 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623460-1652-46d4-983a-e56b2cb03f75 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.608154 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.608238 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d623460-1652-46d4-983a-e56b2cb03f75 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.608238 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 323/661] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:01.608238 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d623460-1652-46d4-983a-e56b2cb03f75 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.620964 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 339/662] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:01.621827 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.709760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d623460-1652-46d4-983a-e56b2cb03f75 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.724544 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab2c22-56a5-4c2d-af0f-cf08b3381500 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.743289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b09e8-8e9b-4c3e-959c-bbbf0e086b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.743973 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b09e8-8e9b-4c3e-959c-bbbf0e086b96 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.744262 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b09e8-8e9b-4c3e-959c-bbbf0e086b96 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.744546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b09e8-8e9b-4c3e-959c-bbbf0e086b96 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.745441 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b09e8-8e9b-4c3e-959c-bbbf0e086b96 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:01.745760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b09e8-8e9b-4c3e-959c-bbbf0e086b96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.747212 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b09e8-8e9b-4c3e-959c-bbbf0e086b96 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.758218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.759016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.759304 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.759580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.769397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 324/663] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:01.769397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f995e8-664b-4944-8c1b-b8708fe87641 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.769397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f995e8-664b-4944-8c1b-b8708fe87641 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.769397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f995e8-664b-4944-8c1b-b8708fe87641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.774240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.775703 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.782844 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-f4f995e8-664b-4944-8c1b-b8708fe87641 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:01.784137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-f4f995e8-664b-4944-8c1b-b8708fe87641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:01.792946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:01.794176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:01.795168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:01.795168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:01.795168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:01.795168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:01.795168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 340/664] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:01.795168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.869146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 325/665] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:01.869146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f995e8-664b-4944-8c1b-b8708fe87641 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.896391 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b6d39-033d-4a37-a879-13c41d5c301f None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:01.929358 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:01.930163 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:01.930450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:01.930729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:01.937359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:01.938828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:01.952897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:01.953941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 341/666] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:01.954891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:01.976922 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f995e8-664b-4944-8c1b-b8708fe87641 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.011489 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7360b-b880-4d71-85de-5f493b29c22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.012623 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7360b-b880-4d71-85de-5f493b29c22e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.013055 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7360b-b880-4d71-85de-5f493b29c22e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.013373 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7360b-b880-4d71-85de-5f493b29c22e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.055091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e3f93-802c-4bbc-b41a-dc58d0d4cbf4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.082098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7360b-b880-4d71-85de-5f493b29c22e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.083655 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7360b-b880-4d71-85de-5f493b29c22e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.094885 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.095570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.095854 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.096130 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.096907 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:02.097306 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.098620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.110526 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 342/667] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:02.115677 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.188182 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 326/668] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:29:02.189537 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7360b-b880-4d71-85de-5f493b29c22e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.215841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e6a46-5ace-4bb3-bcb1-cc526e595f96 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.248360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc34d93-3c66-4547-8ad9-952eb1a34574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.249425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc34d93-3c66-4547-8ad9-952eb1a34574 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.249883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc34d93-3c66-4547-8ad9-952eb1a34574 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.250184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc34d93-3c66-4547-8ad9-952eb1a34574 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.261436 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc34d93-3c66-4547-8ad9-952eb1a34574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-6cc34d93-3c66-4547-8ad9-952eb1a34574 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:02.269880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:02.270976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:02.271877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:02.276218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 343/669] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:02.277342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc34d93-3c66-4547-8ad9-952eb1a34574 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.293320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7360b-b880-4d71-85de-5f493b29c22e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.327586 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.328372 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.328668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.328955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.329805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:02.330123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.332129 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.343827 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 327/670] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:02.344545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.381416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc34d93-3c66-4547-8ad9-952eb1a34574 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.422434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.423232 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.423516 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.423791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.432994 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.434715 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.452268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9e51d-3d10-4f6c-9385-f1e400a53402 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:02.454162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:02.455179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:02.456805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:02.456805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:02.456805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:02.456805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:02.456805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:02.456805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:02.456805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:02.456805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:02.457299 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 344/671] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:02.458169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.491102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.491903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.492189 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.492479 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.503437 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.505090 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:02.524694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:02.525731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:02.526673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:02.526673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:02.526673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:02.526673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:02.526673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 328/672] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:02.526977 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.554584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba75955-4fef-49f6-b8de-49604a269036 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.586998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.587663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.587953 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.588244 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.588998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:02.589352 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.590725 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.598638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 345/673] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:02.599425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.625511 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddd850-882c-4d7f-984c-42b547582762 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.657371 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47797903-2a6f-4e2e-9d6a-57b01f258bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.658024 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47797903-2a6f-4e2e-9d6a-57b01f258bd1 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.658315 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47797903-2a6f-4e2e-9d6a-57b01f258bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.658596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47797903-2a6f-4e2e-9d6a-57b01f258bd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.659467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47797903-2a6f-4e2e-9d6a-57b01f258bd1 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=4e125e49ac664c749fe6eede9531920d) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:02.659786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47797903-2a6f-4e2e-9d6a-57b01f258bd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.661239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47797903-2a6f-4e2e-9d6a-57b01f258bd1 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.666825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 329/674] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/role_assignments?user.id=4e125e49ac664c749fe6eede9531920d&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:02.668166 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50528dd5-762d-465d-b98f-6ae52af6222b None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.668463 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50528dd5-762d-465d-b98f-6ae52af6222b None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.668746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50528dd5-762d-465d-b98f-6ae52af6222b None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.673137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 330/675] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:02.673137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50528dd5-762d-465d-b98f-6ae52af6222b None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.700301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418644e1-6d1d-440a-8b0b-de07282195ac None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.740001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.740899 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.741361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.741769 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.743017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:02.743502 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.745817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.764554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 346/676] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:02.765498 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.791035 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50528dd5-762d-465d-b98f-6ae52af6222b None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.828768 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba36644-d9d5-41a2-a0d5-e821e60e43f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.829657 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba36644-d9d5-41a2-a0d5-e821e60e43f7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.830017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba36644-d9d5-41a2-a0d5-e821e60e43f7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.830361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba36644-d9d5-41a2-a0d5-e821e60e43f7 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.831176 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba36644-d9d5-41a2-a0d5-e821e60e43f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:02.831556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba36644-d9d5-41a2-a0d5-e821e60e43f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.833147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba36644-d9d5-41a2-a0d5-e821e60e43f7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.850672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 331/677] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:02.850672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cfcf5-c402-4ba4-9ab3-39d559279463 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.850672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cfcf5-c402-4ba4-9ab3-39d559279463 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.850672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4cfcf5-c402-4ba4-9ab3-39d559279463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.867875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-6d4cfcf5-c402-4ba4-9ab3-39d559279463 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:02.869202 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-6d4cfcf5-c402-4ba4-9ab3-39d559279463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:02.880248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f27c72-1712-4d05-bea1-ca8fb42e5b75 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:02.920817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:02.921670 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:02.921956 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:02.922233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:02.933654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:02.935421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:02.944401 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 332/678] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:02.946010 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cfcf5-c402-4ba4-9ab3-39d559279463 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:02.955753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:02.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:02.958142 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:02.958142 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:02.958142 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:02.958142 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:02.958142 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 347/679] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:02.958142 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.057391 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d665d3-57aa-4763-8ff4-732728830bc8 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.060460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4cfcf5-c402-4ba4-9ab3-39d559279463 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.093541 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.094456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.094950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.095294 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.100180 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.100916 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.101292 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.101716 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.102686 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=365477e05c1f48beac7f88e6a0707bd6) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:03.103061 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.103761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.104651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:03.105371 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:03.111625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 333/680] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/role_assignments?user.id=365477e05c1f48beac7f88e6a0707bd6&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:03.113142 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:03.130660 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:03.131683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:03.132624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:03.132624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:03.132624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:03.132624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:03.132624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:03.132624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:03.132624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:03.132624 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:03.134844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 348/681] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:03.135737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.239132 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069280e5-fdb5-4d25-a6fd-53e6a360646c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.260672 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08f1fd-4785-4f03-b64d-8c65bbe7204d None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.277299 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.278010 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.278591 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.278877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.279643 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:03.279962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.281432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:03.299370 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 334/682] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:03.300292 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.301916 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6852a5-da11-43c3-896d-25f077b46239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.302839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6852a5-da11-43c3-896d-25f077b46239 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.303211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6852a5-da11-43c3-896d-25f077b46239 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.303581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6852a5-da11-43c3-896d-25f077b46239 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.317974 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6852a5-da11-43c3-896d-25f077b46239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-ae6852a5-da11-43c3-896d-25f077b46239 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:03.326523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:03.327507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:03.328434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 349/683] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:03.329234 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6852a5-da11-43c3-896d-25f077b46239 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.415794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abf135-1d59-47e1-8280-4acc28a0e9a3 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.435248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6852a5-da11-43c3-896d-25f077b46239 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.460088 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.461294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.462103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.462883 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.464072 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:03.464896 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.466651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:03.469324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.470195 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.470545 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.470894 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] PATH_INFO: `/v3/users/nova` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.479619 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.481292 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 335/684] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:03.482622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.486154 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:03.502873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:03.503935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:03.504934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 350/685] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:03.509844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.587887 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715e4e9-7c01-4b2b-9ebf-81565bec14fd None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.611955 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f56556-9727-4827-838b-3dc87c586429 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.622869 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.623857 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.624347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.624841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.625856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:03.626386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.628513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:03.637139 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 336/686] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:03.638228 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.646981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.647647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.647973 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.648256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.649180 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=5382d7219c244430ae6f37d6b32e7d8a) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:03.649494 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.651036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:03.665266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 351/687] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/role_assignments?user.id=5382d7219c244430ae6f37d6b32e7d8a&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:03.666552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.746485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da445f52-a80d-44e6-a2c1-1586c9d557c8 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.782661 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.783722 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.784280 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.784789 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.785939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:03.786643 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.788330 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:03.808443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 337/688] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:03.818748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:03.917135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a430e-6344-489d-8868-e3ae805a7600 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.975609 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:03.976478 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:03.976763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:03.977053 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:03.981701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7335a-d6ba-4f88-8c92-c621e357a551 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:03.990108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:03.991585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:04.016071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:04.017114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:04.018074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:04.018074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:04.018074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:04.018074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:04.018074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 352/689] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:04.019315 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c185018-1570-44a4-be40-80de8127c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:04.020964 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185018-1570-44a4-be40-80de8127c668 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:04.020964 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185018-1570-44a4-be40-80de8127c668 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:04.020964 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c185018-1570-44a4-be40-80de8127c668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:04.023238 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:04.107410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c185018-1570-44a4-be40-80de8127c668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:04.108877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c185018-1570-44a4-be40-80de8127c668 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:04.154375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39239aa5-0e45-4993-84f0-eb86788d41b3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:04.198122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:04.198886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:04.199407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:04.199918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:04.200879 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:04.201233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:04.203263 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:04.218890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 353/690] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:04.225831 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:04.231452 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 338/691] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:29:04.238635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c185018-1570-44a4-be40-80de8127c668 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:04.395075 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0be29-6c48-475e-a534-6c9dbe831d3b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:04.409849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c185018-1570-44a4-be40-80de8127c668 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:04.454759 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:04.455481 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:04.455787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:04.456067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] PATH_INFO: `/v3/users/swift` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:04.464140 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:04.464933 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:04.465244 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:04.465525 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:04.474222 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:04.475684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:04.476780 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:04.478749 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:04.498408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:04.499520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:29:04.500517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:04.512580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:04.513648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:04.514625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:04.514625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:04.514625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:04.514625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:04.521206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 354/692] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:04.523795 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 339/693] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:04.532213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:04.541243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:04.718902 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6fec6-054e-44b7-8913-345b2fe4084a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:04.728680 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1330c61-d402-4e6b-997f-727c6f775cfa None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:04.758015 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80404551-0078-4d22-a806-23d798e176db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:04.759273 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80404551-0078-4d22-a806-23d798e176db None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:04.760939 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80404551-0078-4d22-a806-23d798e176db None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:04.761456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80404551-0078-4d22-a806-23d798e176db None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:04.762421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80404551-0078-4d22-a806-23d798e176db None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:04.763685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80404551-0078-4d22-a806-23d798e176db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:04.767923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80404551-0078-4d22-a806-23d798e176db None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:04.786280 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 355/694] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:04.793976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80404551-0078-4d22-a806-23d798e176db None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:04.797998 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:04.799104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:04.799104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:04.799314 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:04.800597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:04.800909 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:04.802513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:04.840828 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 340/695] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:29:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:04.858621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:04.903197 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80404551-0078-4d22-a806-23d798e176db None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:04.942452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:04.943881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:04.946690 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:04.947289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:04.964016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:04.967084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:04.992737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:04.994162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:04.995097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:04.995097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:04.995097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:04.995097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:04.995097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:04.995097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:04.995097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:04.995097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:04.997708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 356/696] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:05.003323 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:05.086580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398943aa-ef46-4266-be93-d4f69848bfac None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:05.147965 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d335b63-7d49-493b-a9a1-a68d7cbbe53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:05.149073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335b63-7d49-493b-a9a1-a68d7cbbe53e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.149589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335b63-7d49-493b-a9a1-a68d7cbbe53e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.150108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335b63-7d49-493b-a9a1-a68d7cbbe53e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.180590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ca866-5bec-4603-bc76-06bcfddaf671 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:05.223475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d335b63-7d49-493b-a9a1-a68d7cbbe53e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:05.224021 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1ff2b-1b9a-43b6-96f3-7a7ddf97d6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:05.224701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ff2b-1b9a-43b6-96f3-7a7ddf97d6f8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.224991 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ff2b-1b9a-43b6-96f3-7a7ddf97d6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.225301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ff2b-1b9a-43b6-96f3-7a7ddf97d6f8 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.226114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1ff2b-1b9a-43b6-96f3-7a7ddf97d6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:05.226431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1ff2b-1b9a-43b6-96f3-7a7ddf97d6f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:05.226729 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d335b63-7d49-493b-a9a1-a68d7cbbe53e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:05.227884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1ff2b-1b9a-43b6-96f3-7a7ddf97d6f8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:05.244215 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 357/697] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:05.245990 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995d9a-e626-4dcc-9b9a-90eab22c5586 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.247603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995d9a-e626-4dcc-9b9a-90eab22c5586 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.248112 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995d9a-e626-4dcc-9b9a-90eab22c5586 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.250388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 358/698] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:05.253858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05995d9a-e626-4dcc-9b9a-90eab22c5586 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:05.361075 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 341/699] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:29:05.361075 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12191b-a10d-43b8-9aa1-96d9442ff698 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.361075 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12191b-a10d-43b8-9aa1-96d9442ff698 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.361075 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12191b-a10d-43b8-9aa1-96d9442ff698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.368146 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05995d9a-e626-4dcc-9b9a-90eab22c5586 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:05.379854 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-8e12191b-a10d-43b8-9aa1-96d9442ff698 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:05.381461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-8e12191b-a10d-43b8-9aa1-96d9442ff698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:05.402176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:05.402837 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.403137 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.403415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.404244 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=17730dd371184663b2c0eb091cc668a3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:05.404560 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:05.406026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:05.412502 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 359/700] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:05 2022] GET /identity/v3/role_assignments?user.id=17730dd371184663b2c0eb091cc668a3&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=58c34600086c469b87bfa39b74dd2607 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:05.413302 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:05.457973 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 342/701] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:05.487066 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12191b-a10d-43b8-9aa1-96d9442ff698 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:05.609507 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12191b-a10d-43b8-9aa1-96d9442ff698 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:05.632886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47109650-e280-40a1-aab8-c645d5f4f5a0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:05.645224 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:05.646028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.646317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.646596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.657031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:05.658534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:05.671879 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:05.672997 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.673712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.674192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:05.677937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:05.678868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:05.679810 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:05.679810 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:05.679810 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:05.679810 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:05.679810 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 343/702] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:05.689063 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:05.698349 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:05.699998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:05.730743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:05.731887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:05.733432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:05.733432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:05.733432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:05.733432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:05.733432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 360/703] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:05.741187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:05.843730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e16c4-cae2-490b-853a-654eed0349c4 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:05.883948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f906e7-84c2-4c3d-b789-73c3ff7147e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:05.884462 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56827d-72e8-4ade-b575-e12523d286d6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:05.884591 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f906e7-84c2-4c3d-b789-73c3ff7147e0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.884816 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f906e7-84c2-4c3d-b789-73c3ff7147e0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.885128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f906e7-84c2-4c3d-b789-73c3ff7147e0 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.886128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f906e7-84c2-4c3d-b789-73c3ff7147e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:05.886446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f906e7-84c2-4c3d-b789-73c3ff7147e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:05.888757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f906e7-84c2-4c3d-b789-73c3ff7147e0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:05.909128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 344/704] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:05.910916 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1c266-0c5b-44a3-ab22-cf262f168891 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.911222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1c266-0c5b-44a3-ab22-cf262f168891 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.911691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1c266-0c5b-44a3-ab22-cf262f168891 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.914072 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 345/705] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:05.914952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1c266-0c5b-44a3-ab22-cf262f168891 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:05.926764 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb81059-7947-499e-b8b7-095f6e788071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:05.927446 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb81059-7947-499e-b8b7-095f6e788071 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.927731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb81059-7947-499e-b8b7-095f6e788071 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.928008 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb81059-7947-499e-b8b7-095f6e788071 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.928754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb81059-7947-499e-b8b7-095f6e788071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:05.929100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb81059-7947-499e-b8b7-095f6e788071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:05.930497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb81059-7947-499e-b8b7-095f6e788071 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:05.943837 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 361/706] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:05.945295 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745581-13ab-4bab-bb2e-128028620704 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:05.945590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745581-13ab-4bab-bb2e-128028620704 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:05.945873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745581-13ab-4bab-bb2e-128028620704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:05.963113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-b5745581-13ab-4bab-bb2e-128028620704 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:05.964323 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-b5745581-13ab-4bab-bb2e-128028620704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:06.040847 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1c266-0c5b-44a3-ab22-cf262f168891 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:06.041219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 362/707] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:06.042045 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5745581-13ab-4bab-bb2e-128028620704 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:06.078740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bda4da-d12e-4114-9210-708f0ef78fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:06.079419 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda4da-d12e-4114-9210-708f0ef78fe5 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.079708 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda4da-d12e-4114-9210-708f0ef78fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.079995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda4da-d12e-4114-9210-708f0ef78fe5 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.081748 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bda4da-d12e-4114-9210-708f0ef78fe5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:06.083671 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bda4da-d12e-4114-9210-708f0ef78fe5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:06.125098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 346/708] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:29:05 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:06.125098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bda4da-d12e-4114-9210-708f0ef78fe5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:06.244272 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5745581-13ab-4bab-bb2e-128028620704 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:06.283191 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:06.284046 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.284335 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.284612 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.291890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:06.293397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:06.308796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:06.309839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:06.310789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 363/709] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:06.316344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:06.351347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bda4da-d12e-4114-9210-708f0ef78fe5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:06.410989 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:06.412093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.412537 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.412964 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.413813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:06.415344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:06.426887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb89b73-3eff-484c-9815-90cf3c0bba69 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:06.430380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:06.431603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:06.432703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:06.435450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 347/710] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:29:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:06.446322 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:06.468813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774eb3d-67ca-40a6-94ea-ce14c2202951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:06.468813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774eb3d-67ca-40a6-94ea-ce14c2202951 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.469175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774eb3d-67ca-40a6-94ea-ce14c2202951 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.469283 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774eb3d-67ca-40a6-94ea-ce14c2202951 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.470064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774eb3d-67ca-40a6-94ea-ce14c2202951 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:06.470381 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774eb3d-67ca-40a6-94ea-ce14c2202951 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:06.471860 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774eb3d-67ca-40a6-94ea-ce14c2202951 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:06.490398 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 364/711] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:06.491819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a59f3-ac6d-4756-9b51-7ee760de4463 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.492116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a59f3-ac6d-4756-9b51-7ee760de4463 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.492404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a59f3-ac6d-4756-9b51-7ee760de4463 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.493938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 365/712] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:06.495707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a59f3-ac6d-4756-9b51-7ee760de4463 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:06.570333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c35af1-b71f-41ae-9712-58bec9e90436 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:06.604882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc544e19-c392-4b81-b982-c3fdba65883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:06.605589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc544e19-c392-4b81-b982-c3fdba65883d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.605875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc544e19-c392-4b81-b982-c3fdba65883d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.606389 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc544e19-c392-4b81-b982-c3fdba65883d None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.606867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc544e19-c392-4b81-b982-c3fdba65883d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:06.607443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc544e19-c392-4b81-b982-c3fdba65883d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:06.608825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc544e19-c392-4b81-b982-c3fdba65883d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:06.614269 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 348/713] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:29:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:06.614615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a59f3-ac6d-4756-9b51-7ee760de4463 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:06.619721 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e502e-5eb4-446a-8670-74e21939bfd4 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.619721 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e502e-5eb4-446a-8670-74e21939bfd4 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.619721 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e502e-5eb4-446a-8670-74e21939bfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.639107 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-f14e502e-5eb4-446a-8670-74e21939bfd4 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:06.640437 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-f14e502e-5eb4-446a-8670-74e21939bfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:06.669795 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:06.670566 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.670922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.671268 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.672701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=a752b7105b754628a153e396a9110653) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:06.673121 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:06.675699 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:06.684663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 366/714] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:06 2022] GET /identity/v3/role_assignments?user.id=a752b7105b754628a153e396a9110653&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:06.689138 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:06.764354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 349/715] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:06.798736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e502e-5eb4-446a-8670-74e21939bfd4 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:06.805065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10adfa6-45c6-438c-a081-1b68cf31cdc6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:06.843516 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:06.844279 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.844566 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.844843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.845817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:06.846135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:06.848623 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:06.856211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 367/716] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:29:06 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:06.954316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e502e-5eb4-446a-8670-74e21939bfd4 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:06.995417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:06.996547 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:06.996981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:06.997431 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] PATH_INFO: `/v3/services/network` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:06.998285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:06.999829 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:07.013070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:07.014484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:07.015578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:07.018343 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 350/717] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:06 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:07.024465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:07.136176 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90dfa-09cd-4948-be0f-6476b73ce607 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:07.175508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace4f20-f2ff-44e9-886f-92954722961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:07.176468 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4f20-f2ff-44e9-886f-92954722961e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:07.176962 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4f20-f2ff-44e9-886f-92954722961e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:07.177406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4f20-f2ff-44e9-886f-92954722961e None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:07.178320 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace4f20-f2ff-44e9-886f-92954722961e None admin] RBAC: Adding query filter params (name=network) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:07.178793 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace4f20-f2ff-44e9-886f-92954722961e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:07.180321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace4f20-f2ff-44e9-886f-92954722961e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:07.186200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 368/718] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:07.194117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:07.319560 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda6524-14d3-4266-95b1-431ba3697424 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:07.369918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1627da9-c71e-4398-bd1f-969afe42819f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:07.370908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627da9-c71e-4398-bd1f-969afe42819f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:07.371343 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627da9-c71e-4398-bd1f-969afe42819f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:07.371780 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627da9-c71e-4398-bd1f-969afe42819f None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:07.372653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627da9-c71e-4398-bd1f-969afe42819f None admin] RBAC: Adding query filter params (type=network) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:07.373144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627da9-c71e-4398-bd1f-969afe42819f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:07.374655 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1627da9-c71e-4398-bd1f-969afe42819f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:07.380607 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 351/719] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:07.950763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c7ad9-7de7-406a-97ce-9c1e969dcdbd None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:07.951110 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c7ad9-7de7-406a-97ce-9c1e969dcdbd None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:07.951472 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c7ad9-7de7-406a-97ce-9c1e969dcdbd None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:07.953008 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 369/720] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:07.958583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34526c5-0cc1-4884-9431-5cef031e9821 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:07.958883 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34526c5-0cc1-4884-9431-5cef031e9821 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:07.959172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34526c5-0cc1-4884-9431-5cef031e9821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:07.986051 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-d34526c5-0cc1-4884-9431-5cef031e9821 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:07.987812 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-d34526c5-0cc1-4884-9431-5cef031e9821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:08.074742 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 352/721] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:29:08.114685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c7ad9-7de7-406a-97ce-9c1e969dcdbd None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:08.243869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c7ad9-7de7-406a-97ce-9c1e969dcdbd None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:08.293782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9d780-41a9-45de-8150-477eb625bb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:08.294596 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9d780-41a9-45de-8150-477eb625bb83 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:08.294885 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9d780-41a9-45de-8150-477eb625bb83 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:08.295165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9d780-41a9-45de-8150-477eb625bb83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:08.310212 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9d780-41a9-45de-8150-477eb625bb83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-8ec9d780-41a9-45de-8150-477eb625bb83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:08.318271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:08.319338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:08.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:08.325290 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34526c5-0cc1-4884-9431-5cef031e9821 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:08.327282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 370/722] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:08.463007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34526c5-0cc1-4884-9431-5cef031e9821 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:08.513991 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:08.514614 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:08.514908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:08.515198 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:08.515982 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:08.516299 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:08.517736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:08.529707 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 353/723] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:08.535468 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9d780-41a9-45de-8150-477eb625bb83 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:08.670443 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9d780-41a9-45de-8150-477eb625bb83 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:08.713524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fc6cc-c8ad-4fc4-a530-69a4dc952d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:08.714334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc6cc-c8ad-4fc4-a530-69a4dc952d15 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:08.714620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc6cc-c8ad-4fc4-a530-69a4dc952d15 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:08.714906 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fc6cc-c8ad-4fc4-a530-69a4dc952d15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:08.739138 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fc6cc-c8ad-4fc4-a530-69a4dc952d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:08.740901 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fc6cc-c8ad-4fc4-a530-69a4dc952d15 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-4b0fc6cc-c8ad-4fc4-a530-69a4dc952d15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:08.760881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:08.763254 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:08.765774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:08.765774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:08.765774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:08.765774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:08.765774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 371/724] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:08.767959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.018694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f51bd-107d-4428-aa9a-48f980c325d1 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:09.052391 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d53e0-d096-482e-8638-2857c5336c30 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.052730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d53e0-d096-482e-8638-2857c5336c30 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.052921 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d53e0-d096-482e-8638-2857c5336c30 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.054334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 372/725] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:09.061580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:09.062222 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6f1b3-c8d7-4706-9770-13cfe76aaed8 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.062419 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.062584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6f1b3-c8d7-4706-9770-13cfe76aaed8 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.062768 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.062929 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6f1b3-c8d7-4706-9770-13cfe76aaed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.063085 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.063521 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:09.063834 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:09.065454 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:09.078803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 354/726] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:09.086450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.096094 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-4ea6f1b3-c8d7-4706-9770-13cfe76aaed8 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:09.096917 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-4ea6f1b3-c8d7-4706-9770-13cfe76aaed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:09.226590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97930500-b2b3-41c1-9f9c-f85f4da32f58 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:09.263665 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 373/727] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:09.276349 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:09.277117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.277410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.277705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] PATH_INFO: `/v3/users/nova` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.288221 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6f1b3-c8d7-4706-9770-13cfe76aaed8 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.290318 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:09.292036 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:09.314996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:09.316120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:09.317137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:09.319966 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 355/728] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:09.325378 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.391968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6f1b3-c8d7-4706-9770-13cfe76aaed8 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:09.436976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d36e4-466a-46f6-9438-e74eace35736 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:09.441774 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3f38-dbc8-49cd-b59e-1532dddd0311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:09.442597 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3f38-dbc8-49cd-b59e-1532dddd0311 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.442892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3f38-dbc8-49cd-b59e-1532dddd0311 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.443170 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3f38-dbc8-49cd-b59e-1532dddd0311 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.462482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf3f38-dbc8-49cd-b59e-1532dddd0311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-06bf3f38-dbc8-49cd-b59e-1532dddd0311 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:09.474361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:09.475375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:09.476269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:09.480401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 374/729] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:09.483541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86050775-cf20-4c28-94ba-ce409b94cf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:09.484533 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86050775-cf20-4c28-94ba-ce409b94cf90 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.485059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86050775-cf20-4c28-94ba-ce409b94cf90 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.485553 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86050775-cf20-4c28-94ba-ce409b94cf90 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.486602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3f38-dbc8-49cd-b59e-1532dddd0311 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.488082 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86050775-cf20-4c28-94ba-ce409b94cf90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:09.489056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86050775-cf20-4c28-94ba-ce409b94cf90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:09.492525 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86050775-cf20-4c28-94ba-ce409b94cf90 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:09.517865 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 356/730] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:09.520218 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571982-59b3-4e35-9b5d-e513c831092d None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.520513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571982-59b3-4e35-9b5d-e513c831092d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.520794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2571982-59b3-4e35-9b5d-e513c831092d None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.526346 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 357/731] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:09.527247 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2571982-59b3-4e35-9b5d-e513c831092d None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.607616 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3f38-dbc8-49cd-b59e-1532dddd0311 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:09.642735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e47b79-afa0-477c-b52d-74fa566fcfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:09.644386 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e47b79-afa0-477c-b52d-74fa566fcfe1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.644985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e47b79-afa0-477c-b52d-74fa566fcfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.645535 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e47b79-afa0-477c-b52d-74fa566fcfe1 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.646503 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2571982-59b3-4e35-9b5d-e513c831092d None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:09.646662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e47b79-afa0-477c-b52d-74fa566fcfe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:09.647118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e47b79-afa0-477c-b52d-74fa566fcfe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:09.648684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e47b79-afa0-477c-b52d-74fa566fcfe1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:09.657778 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 375/732] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:09.659502 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3240335-a16b-4175-a83c-f9d169a4490f None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.660037 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3240335-a16b-4175-a83c-f9d169a4490f None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.660542 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3240335-a16b-4175-a83c-f9d169a4490f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.677511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-f3240335-a16b-4175-a83c-f9d169a4490f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:09.679236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-f3240335-a16b-4175-a83c-f9d169a4490f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:09.681865 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:09.682670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.682956 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.683232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.694470 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:09.695937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:09.715172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:09.716200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:09.717170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:09.717170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:09.717170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:09.717170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:09.717170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 358/733] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:09.718137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.756122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 376/734] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:09.757372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3240335-a16b-4175-a83c-f9d169a4490f None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.847793 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f1021-a7f5-422b-ae70-9ce06823b214 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:09.861096 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3240335-a16b-4175-a83c-f9d169a4490f None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:09.883653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:09.884625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.885023 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.885440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.897634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:09.898623 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:09.899493 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:09.899734 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:09.900186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:09.900467 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:09.901325 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:09.901821 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:09.903251 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:09.916656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 377/735] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:09.917648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:09.922080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:09.923115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:09.924079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:09.924079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:09.924079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:09.924079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:09.927525 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 359/736] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:09.928853 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.020868 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258714-1526-4eb3-bc4f-943de993e6e7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.031345 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919e23b-2d8f-44be-90f3-70d4163e3a18 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.055391 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c8db8-0424-4ee7-99e1-0b78f9002a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.056281 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8db8-0424-4ee7-99e1-0b78f9002a1a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.056635 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8db8-0424-4ee7-99e1-0b78f9002a1a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.056975 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8db8-0424-4ee7-99e1-0b78f9002a1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.067949 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.068223 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c8db8-0424-4ee7-99e1-0b78f9002a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:10.069243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.069726 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.070219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-132c8db8-0424-4ee7-99e1-0b78f9002a1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:10.076397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:10.077431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:10.078354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:10.080550 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.081452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 378/737] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:10.082346 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c8db8-0424-4ee7-99e1-0b78f9002a1a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.082880 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:10.098115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:10.099060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:10.100015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:10.100015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:10.100015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:10.100015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:10.100015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:10.100015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:10.100015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:10.100015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:10.102785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 360/738] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:10.103983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.203909 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c8db8-0424-4ee7-99e1-0b78f9002a1a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.207316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d673d98-7318-4a31-950c-e2a9ff2caee5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.238374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.239446 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.240036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.240511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.240624 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f256c61-e6e0-4031-a0fb-c4a853236f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.241287 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f256c61-e6e0-4031-a0fb-c4a853236f33 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.241576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f256c61-e6e0-4031-a0fb-c4a853236f33 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.241856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f256c61-e6e0-4031-a0fb-c4a853236f33 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.242230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:10.242616 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f256c61-e6e0-4031-a0fb-c4a853236f33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:10.242934 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f256c61-e6e0-4031-a0fb-c4a853236f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.243103 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.244310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f256c61-e6e0-4031-a0fb-c4a853236f33 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.244848 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.252186 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 361/739] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:10.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3a9d-2fc7-41f7-99f7-40d9c5aee25e None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3a9d-2fc7-41f7-99f7-40d9c5aee25e None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf3a9d-2fc7-41f7-99f7-40d9c5aee25e None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.257415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 362/740] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:10.259116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf3a9d-2fc7-41f7-99f7-40d9c5aee25e None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.261986 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 379/741] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:10.263215 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.361350 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b20bb1-6829-4dfd-aa8d-46d79cfa8cdd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.362800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf3a9d-2fc7-41f7-99f7-40d9c5aee25e None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.393558 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.393850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc80d8a-49a3-4942-a2dd-e44e867303dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.394346 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.394502 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80d8a-49a3-4942-a2dd-e44e867303dc None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.394632 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.394791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80d8a-49a3-4942-a2dd-e44e867303dc None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.394898 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.395069 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80d8a-49a3-4942-a2dd-e44e867303dc None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.395858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc80d8a-49a3-4942-a2dd-e44e867303dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:10.396169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc80d8a-49a3-4942-a2dd-e44e867303dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.397630 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc80d8a-49a3-4942-a2dd-e44e867303dc None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.404633 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.406104 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.409556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 363/742] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:10.410735 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81549-a11d-4f88-ad5c-5f1c05869194 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.411025 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81549-a11d-4f88-ad5c-5f1c05869194 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.411305 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc81549-a11d-4f88-ad5c-5f1c05869194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:10.424106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:10.425746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:10.426689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:10.426689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:10.426689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:10.426689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:10.426689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 380/743] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:10.426689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.430589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-9fc81549-a11d-4f88-ad5c-5f1c05869194 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:10.431778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-9fc81549-a11d-4f88-ad5c-5f1c05869194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:10.495111 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 364/744] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:10.496409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2425d8-89b5-4997-bc5f-6f577ac81605 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.496696 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2425d8-89b5-4997-bc5f-6f577ac81605 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.496971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2425d8-89b5-4997-bc5f-6f577ac81605 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.501077 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 365/745] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:10.501077 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2425d8-89b5-4997-bc5f-6f577ac81605 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.523417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588799b-f7af-4e8b-9da9-624c3d35a519 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.556963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.557798 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.558257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.558541 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.565396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.566844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:10.581452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:10.582415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 381/746] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:29:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:10.583399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.611972 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2425d8-89b5-4997-bc5f-6f577ac81605 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.644168 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01a5f9-a78d-4a54-b237-4732221fc441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.645060 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01a5f9-a78d-4a54-b237-4732221fc441 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.645364 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01a5f9-a78d-4a54-b237-4732221fc441 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.645637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01a5f9-a78d-4a54-b237-4732221fc441 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/17730dd371184663b2c0eb091cc668a3/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.647607 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01a5f9-a78d-4a54-b237-4732221fc441 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=17730dd371184663b2c0eb091cc668a3, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.649052 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01a5f9-a78d-4a54-b237-4732221fc441 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.661268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 366/747] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:29:10 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/17730dd371184663b2c0eb091cc668a3/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:29:10.662673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a3d7-6f99-4870-be1b-9658a807ecc8 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.662962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a3d7-6f99-4870-be1b-9658a807ecc8 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.663236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a3d7-6f99-4870-be1b-9658a807ecc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.681104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-f899a3d7-6f99-4870-be1b-9658a807ecc8 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:10.682361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-f899a3d7-6f99-4870-be1b-9658a807ecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:10.688422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480df475-4e62-4eae-b9cc-5131d63461d3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.729462 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.730150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.730441 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.730723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.731472 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:10.731789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.733544 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.750733 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 382/748] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:10.750733 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.761197 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 367/749] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:29:10.762226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899a3d7-6f99-4870-be1b-9658a807ecc8 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.865543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899a3d7-6f99-4870-be1b-9658a807ecc8 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.869492 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c306002-d815-453e-a61a-a03f8f4e65b5 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:10.900393 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.901407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.902005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.902275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a3c05-d438-4048-8dc0-deb7cc90a529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:10.902703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.902944 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3c05-d438-4048-8dc0-deb7cc90a529 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:10.903227 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3c05-d438-4048-8dc0-deb7cc90a529 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:10.903503 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3c05-d438-4048-8dc0-deb7cc90a529 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:10.904265 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a3c05-d438-4048-8dc0-deb7cc90a529 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.904505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:10.905005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:10.905684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a3c05-d438-4048-8dc0-deb7cc90a529 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.906875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:10.926874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 383/750] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:29:10 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:10.926874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a3c05-d438-4048-8dc0-deb7cc90a529 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:10.927449 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 368/751] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:29:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:10.928310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:11.054670 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a3c05-d438-4048-8dc0-deb7cc90a529 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:11.060692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f7c5c-292e-4a44-9272-3c844e915db7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:11.095144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:11.095969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:11.096288 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:11.099018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] PATH_INFO: `/v3/users/swift` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:11.108707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:11.110250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:11.110382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75257239-a0e7-4b11-8863-99ce8148f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:11.111059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75257239-a0e7-4b11-8863-99ce8148f32c None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:11.111344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75257239-a0e7-4b11-8863-99ce8148f32c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:11.111620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75257239-a0e7-4b11-8863-99ce8148f32c None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:11.113331 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75257239-a0e7-4b11-8863-99ce8148f32c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:11.114703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75257239-a0e7-4b11-8863-99ce8148f32c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:11.126840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:11.128582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:29:11.129637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:11.131587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 384/752] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:11.132893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:11.133230 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 369/753] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:29:10 2022] POST /identity/v3/services => generated 240 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:11.138164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75257239-a0e7-4b11-8863-99ce8148f32c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:11.261236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6da97-33d3-4bef-a00b-f5d57871b077 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:11.308554 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75257239-a0e7-4b11-8863-99ce8148f32c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:11.309993 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812fb7-d08e-476e-9c5f-249b451409f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:11.311106 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812fb7-d08e-476e-9c5f-249b451409f2 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:11.312571 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812fb7-d08e-476e-9c5f-249b451409f2 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:11.313160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812fb7-d08e-476e-9c5f-249b451409f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:11.353412 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:11.354093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:11.354376 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:11.354655 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:11.355514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:11.355827 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:11.357606 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:11.375636 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 370/754] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:29:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:11.383086 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:11.398166 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba812fb7-d08e-476e-9c5f-249b451409f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:11.399688 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba812fb7-d08e-476e-9c5f-249b451409f2 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:11.488256 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b238172-66c2-43a9-b1d3-856260c55554 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:11.500159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 385/755] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:11.506536 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812fb7-d08e-476e-9c5f-249b451409f2 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:11.536245 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fc53c-c414-4526-847e-91c734e06783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:11.536906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc53c-c414-4526-847e-91c734e06783 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:11.537218 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc53c-c414-4526-847e-91c734e06783 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:11.537497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc53c-c414-4526-847e-91c734e06783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:11.610345 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fc53c-c414-4526-847e-91c734e06783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:11.611809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fc53c-c414-4526-847e-91c734e06783 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:11.715054 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 371/756] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:11.720425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fc53c-c414-4526-847e-91c734e06783 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:11.741837 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812fb7-d08e-476e-9c5f-249b451409f2 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:11.815695 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:11.816505 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:11.816791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:11.817099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:11.839504 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:11.840984 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:11.854048 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fc53c-c414-4526-847e-91c734e06783 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:11.863995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:11.865066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:11.866129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:11.866129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:11.866129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:11.866129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:11.866399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 386/757] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:11.878192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:11.901098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:11.901903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:11.902186 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:11.902462 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:11.913407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:11.914865 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:11.933207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:11.936638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:11.937849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:11.937849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:11.937849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:11.937849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:11.937849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 372/758] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:11.946326 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:12.038667 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bce48-3546-430a-8f22-56c9e82ca6fe None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:12.111662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:12.114151 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:12.114151 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:12.114151 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:12.114673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:12.114673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:12.115855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:12.137074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cff32b-d2d9-449f-bbd2-e789564bdb5a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:12.138629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 387/759] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:12.144184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:12.179621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:12.180304 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:12.180631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:12.180917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:12.181681 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:12.181990 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:12.183364 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:12.206821 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 373/760] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:12.212223 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:12.255951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67376d-6f1d-4889-8fd0-b4eb9daa30f3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:12.298090 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:12.299201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:12.299683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:12.300131 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:12.321423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:12.322888 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:12.336018 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad22e5-daed-4a9a-a508-035f48142713 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:12.345343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:12.348082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:12.350359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:12.350359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:12.350359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:12.350359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:12.350359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:12.350359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:12.350359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:12.350359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:12.357202 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 388/761] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:12.359105 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:12.380142 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:12.380955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:12.381261 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:12.381536 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:12.388571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:12.390094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:12.404360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:12.405573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:12.406573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 374/762] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:12.417155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:12.527222 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80807f56-b388-427a-ac61-feadf09323f4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:12.565784 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc572856-39e9-4f4c-abaa-c7af2496fc34 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:12.600019 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:12.600717 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:12.600996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:12.601302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:12.602087 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:12.602396 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:12.609069 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:12.616284 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:12.621076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 375/763] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:12.621369 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:12.622703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:12.623135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:12.623431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:12.624266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:12.624576 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:12.626065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:12.657600 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 389/764] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:12.663200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:12.731555 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a8a3-de04-4a70-8635-65bb2397d5b2 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:12.779122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48651cb8-9454-4c4e-bba3-097bdb784394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:12.779850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651cb8-9454-4c4e-bba3-097bdb784394 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:12.780137 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651cb8-9454-4c4e-bba3-097bdb784394 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:12.780414 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48651cb8-9454-4c4e-bba3-097bdb784394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:12.781308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48651cb8-9454-4c4e-bba3-097bdb784394 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=a752b7105b754628a153e396a9110653) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:12.781626 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48651cb8-9454-4c4e-bba3-097bdb784394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:12.783090 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48651cb8-9454-4c4e-bba3-097bdb784394 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:12.799495 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 376/765] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:12 2022] GET /identity/v3/role_assignments?user.id=a752b7105b754628a153e396a9110653&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=58c34600086c469b87bfa39b74dd2607 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:12.895087 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426c0fe-20a9-4083-b232-65fbc8958d55 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:12.939180 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487cf0b-4ba6-4cbe-833a-9d29e4bffec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:12.939869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487cf0b-4ba6-4cbe-833a-9d29e4bffec6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:12.940710 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487cf0b-4ba6-4cbe-833a-9d29e4bffec6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:12.940993 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487cf0b-4ba6-4cbe-833a-9d29e4bffec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:12.941956 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487cf0b-4ba6-4cbe-833a-9d29e4bffec6 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=8db5070e21174e5593f244bdb2c52224) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:12.942282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487cf0b-4ba6-4cbe-833a-9d29e4bffec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:12.943784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487cf0b-4ba6-4cbe-833a-9d29e4bffec6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:12.957195 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 390/766] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:12 2022] GET /identity/v3/role_assignments?user.id=8db5070e21174e5593f244bdb2c52224&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:13.266856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f2ad-3b10-4560-a124-7bc5b8f6ad76 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:13.267249 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f2ad-3b10-4560-a124-7bc5b8f6ad76 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:13.267433 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6f2ad-3b10-4560-a124-7bc5b8f6ad76 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:13.269115 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 377/767] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:13.278792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d8417-86be-4ee6-827c-28dcc924bc83 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:13.279075 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d8417-86be-4ee6-827c-28dcc924bc83 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:13.279345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d8417-86be-4ee6-827c-28dcc924bc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:13.296267 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-da0d8417-86be-4ee6-827c-28dcc924bc83 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:13.297541 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-da0d8417-86be-4ee6-827c-28dcc924bc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:13.378802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 391/768] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:13.402631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6f2ad-3b10-4560-a124-7bc5b8f6ad76 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:13.554013 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6f2ad-3b10-4560-a124-7bc5b8f6ad76 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:13.597647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950824e-dc0e-4426-88fb-983bc47965f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:13.598472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950824e-dc0e-4426-88fb-983bc47965f9 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:13.598764 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950824e-dc0e-4426-88fb-983bc47965f9 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:13.599035 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950824e-dc0e-4426-88fb-983bc47965f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:13.621990 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950824e-dc0e-4426-88fb-983bc47965f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-e950824e-dc0e-4426-88fb-983bc47965f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:13.630000 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:13.633753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:13.637733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 378/769] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:13.646669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d8417-86be-4ee6-827c-28dcc924bc83 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:13.773776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d8417-86be-4ee6-827c-28dcc924bc83 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:13.774631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc81f83-409e-41ef-ae8b-d83339cb0862 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:13.774920 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc81f83-409e-41ef-ae8b-d83339cb0862 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:13.775205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc81f83-409e-41ef-ae8b-d83339cb0862 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:13.776710 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 379/770] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:13.781708 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f74-4f8b-4fd1-828f-93fc1615355e None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:13.781994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f74-4f8b-4fd1-828f-93fc1615355e None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:13.782272 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd3f74-4f8b-4fd1-828f-93fc1615355e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:13.813164 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:13.813164 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:13.813164 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:13.813164 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:13.813598 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:13.813598 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:13.814400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:13.824594 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 392/771] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:13.825920 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-e4fd3f74-4f8b-4fd1-828f-93fc1615355e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:13.827398 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-e4fd3f74-4f8b-4fd1-828f-93fc1615355e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:13.831510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:13.951014 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 380/772] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:13.974887 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3f74-4f8b-4fd1-828f-93fc1615355e None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.010228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148f7e2-741f-4b2f-b3c8-e365214c3b55 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.070868 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad9833-6876-4f0d-bdbb-a42e1971f96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.071589 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad9833-6876-4f0d-bdbb-a42e1971f96a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.071875 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad9833-6876-4f0d-bdbb-a42e1971f96a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.072156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad9833-6876-4f0d-bdbb-a42e1971f96a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.085757 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad9833-6876-4f0d-bdbb-a42e1971f96a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.087228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad9833-6876-4f0d-bdbb-a42e1971f96a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-d8ad9833-6876-4f0d-bdbb-a42e1971f96a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:14.105316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:14.106257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:14.107469 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:14.107469 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:14.107469 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:14.107469 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:14.111157 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 393/773] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:14.112604 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0974-cdad-4d3d-ba94-b4911b84e95f None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.112909 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0974-cdad-4d3d-ba94-b4911b84e95f None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.113219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a0974-cdad-4d3d-ba94-b4911b84e95f None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.115371 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 394/774] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:14.116315 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0974-cdad-4d3d-ba94-b4911b84e95f None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.142438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd3f74-4f8b-4fd1-828f-93fc1615355e None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.178189 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb07f4-6542-41d4-865f-5a0f51a58697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.179098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb07f4-6542-41d4-865f-5a0f51a58697 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.179379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb07f4-6542-41d4-865f-5a0f51a58697 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.179658 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb07f4-6542-41d4-865f-5a0f51a58697 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.180320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb07f4-6542-41d4-865f-5a0f51a58697 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.181722 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb07f4-6542-41d4-865f-5a0f51a58697 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-f2cb07f4-6542-41d4-865f-5a0f51a58697 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:14.194300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:14.195323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:14.196468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 381/775] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:29:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:14.197622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e8dc6-3c91-4197-af93-8445523abf56 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.201356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e8dc6-3c91-4197-af93-8445523abf56 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.201667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e8dc6-3c91-4197-af93-8445523abf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.220362 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-fa0e8dc6-3c91-4197-af93-8445523abf56 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:14.222053 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-fa0e8dc6-3c91-4197-af93-8445523abf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:14.234103 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a0974-cdad-4d3d-ba94-b4911b84e95f None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.266994 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.267675 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.267958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.268817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.269900 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:14.269900 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.271330 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.288372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 395/776] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:14.288372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.300166 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 382/777] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:14.301285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e8dc6-3c91-4197-af93-8445523abf56 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.393015 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07db1d1-00b4-4b82-9866-ee711df464e8 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.415009 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e8dc6-3c91-4197-af93-8445523abf56 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.428638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.429552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.429998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.430427 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.431319 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:14.431786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.433344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.438542 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 396/778] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:14.439641 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.458799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.459774 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.460067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.460349 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.469691 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.471786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:14.490783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:14.491842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:14.492952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:14.496899 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 383/779] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:14.497765 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.551983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfd889-3e15-480d-a941-7efe9d6bd192 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.593597 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.594395 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.595084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.595376 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] PATH_INFO: `/v3/services/network` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.596048 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.597829 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:14.616482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:14.617664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:14.618578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:14.620375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f7663-a226-4cca-92d5-63ae60bb42fe None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.623722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 397/780] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:14.625377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.663182 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.664794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.665299 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.665809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.666886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:14.668136 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.669805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.677069 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 384/781] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:14.678215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.748536 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed1a2-b7b8-49e4-aba8-fdd896420bcb None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.783471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.784248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.784537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.784813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.785924 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:14.786042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.787480 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.804093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf1a93-7297-471d-8069-dff26163b2d7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.810421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 398/782] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:14.811321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.844442 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.845145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.845456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.845943 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.846963 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] RBAC: Adding query filter params (name=network) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:14.847275 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.848645 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.857179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 385/783] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:14.857179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.923146 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5aaaa-1aab-4e8e-9040-a6bd9dbb3049 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:14.957687 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:14.958356 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:14.958668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:14.958966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:14.959721 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] RBAC: Adding query filter params (interface=public, service_id=fabc1470c4ce428988e4f86d477625fd, region_id=RegionOne) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:14.960041 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:14.961501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:14.971052 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 399/784] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/endpoints?service_id=fabc1470c4ce428988e4f86d477625fd&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:14.972456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:14.981781 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5f4e6-0966-4093-a554-57a28ca38e3a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:15.021250 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d230-ae95-4298-b9fc-68ff12306f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:15.021622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d230-ae95-4298-b9fc-68ff12306f8f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.021830 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d230-ae95-4298-b9fc-68ff12306f8f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.022110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d230-ae95-4298-b9fc-68ff12306f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.093003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b73a0-5cc8-45cf-a8cb-45cfbe1e661c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:15.112592 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48d230-ae95-4298-b9fc-68ff12306f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:15.114768 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48d230-ae95-4298-b9fc-68ff12306f8f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:15.131659 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:15.132488 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.132881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.133293 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.134423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] RBAC: Adding query filter params (type=network) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:15.134423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:15.135972 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:15.145329 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 400/785] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:15.145329 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:15.250784 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 386/786] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:15.251606 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d230-ae95-4298-b9fc-68ff12306f8f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:15.266180 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854accc4-9874-4a24-abef-a86f8a6b4566 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:15.310420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed08e03-c1d4-42ad-836f-48068978f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:15.311718 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08e03-c1d4-42ad-836f-48068978f3e8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.312410 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08e03-c1d4-42ad-836f-48068978f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.312844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08e03-c1d4-42ad-836f-48068978f3e8 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.313904 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed08e03-c1d4-42ad-836f-48068978f3e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:15.316052 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed08e03-c1d4-42ad-836f-48068978f3e8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:15.323775 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 401/787] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:29:15 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:15.324783 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed08e03-c1d4-42ad-836f-48068978f3e8 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:15.359160 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d230-ae95-4298-b9fc-68ff12306f8f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:15.405579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:15.406095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.406385 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.406666 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.407476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] RBAC: Adding query filter params (interface=public, service_id=697a8164f5ec4583a92288de32296df1, region_id=RegionOne) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:15.407794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:15.409825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:15.417998 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 387/788] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:29:15 2022] GET /identity/v3/endpoints?service_id=697a8164f5ec4583a92288de32296df1&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:15.430189 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:15.450818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed08e03-c1d4-42ad-836f-48068978f3e8 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:15.496372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2a379-1b42-478d-b22b-335ca42ddf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:15.497788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a379-1b42-478d-b22b-335ca42ddf09 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.498238 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a379-1b42-478d-b22b-335ca42ddf09 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.498668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2a379-1b42-478d-b22b-335ca42ddf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.513879 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2a379-1b42-478d-b22b-335ca42ddf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:15.516818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2a379-1b42-478d-b22b-335ca42ddf09 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-43c2a379-1b42-478d-b22b-335ca42ddf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:15.542261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:15.543493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:15.544676 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:15.544676 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:15.544676 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:15.544676 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:15.546493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 402/789] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:15.547838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fc2d6-c553-4812-b44a-72eb91820d3e None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.548127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fc2d6-c553-4812-b44a-72eb91820d3e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.548409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fc2d6-c553-4812-b44a-72eb91820d3e None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.554105 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 403/790] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:15.554968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fc2d6-c553-4812-b44a-72eb91820d3e None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:15.604372 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768fdb1-bebc-43cb-937d-557d6b537768 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:15.650124 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a59f9-348b-4b06-9d05-cf0b6ca31534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:15.651311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a59f9-348b-4b06-9d05-cf0b6ca31534 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.652001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a59f9-348b-4b06-9d05-cf0b6ca31534 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.652496 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a59f9-348b-4b06-9d05-cf0b6ca31534 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.654025 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a59f9-348b-4b06-9d05-cf0b6ca31534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:15.655799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a59f9-348b-4b06-9d05-cf0b6ca31534 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:15.668901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 388/791] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:29:15 2022] GET /identity/v3/services => generated 772 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:15.668901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534d95-1be0-4ad3-b110-496a154eecc2 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.668901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534d95-1be0-4ad3-b110-496a154eecc2 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.668901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534d95-1be0-4ad3-b110-496a154eecc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.684184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fc2d6-c553-4812-b44a-72eb91820d3e None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:15.691309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-95534d95-1be0-4ad3-b110-496a154eecc2 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:15.692557 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-95534d95-1be0-4ad3-b110-496a154eecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:15.730001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79390cad-e9a3-40ec-b057-b8805498b536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:15.731022 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79390cad-e9a3-40ec-b057-b8805498b536 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.731314 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79390cad-e9a3-40ec-b057-b8805498b536 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.731762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79390cad-e9a3-40ec-b057-b8805498b536 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.733206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79390cad-e9a3-40ec-b057-b8805498b536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:15.733848 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79390cad-e9a3-40ec-b057-b8805498b536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:15.735622 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79390cad-e9a3-40ec-b057-b8805498b536 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:15.752056 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 404/792] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:15.754250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61956fb0-67d4-4508-8388-739fa80658d8 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.754995 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61956fb0-67d4-4508-8388-739fa80658d8 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.755280 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61956fb0-67d4-4508-8388-739fa80658d8 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.757249 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 405/793] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:15.757850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61956fb0-67d4-4508-8388-739fa80658d8 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:15.786866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 389/794] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:15.788804 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129616-6ba7-41ef-8950-2bb10211a5a2 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.789128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129616-6ba7-41ef-8950-2bb10211a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.789416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba129616-6ba7-41ef-8950-2bb10211a5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.819774 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-ba129616-6ba7-41ef-8950-2bb10211a5a2 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:15.821064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-ba129616-6ba7-41ef-8950-2bb10211a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:15.898657 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 390/795] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:15.899864 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba129616-6ba7-41ef-8950-2bb10211a5a2 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:15.913913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61956fb0-67d4-4508-8388-739fa80658d8 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:15.962735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:15.964371 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:15.964881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:15.965573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:15.974846 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:15.976520 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:15.998230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:15.999415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:16.000580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:16.000580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:16.000580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:16.000580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:16.000580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:16.000580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:16.000580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:16.000580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:16.005141 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 406/796] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:16.006129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.010942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba129616-6ba7-41ef-8950-2bb10211a5a2 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.055214 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e34903-5217-43ba-b007-c1102a571104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.055987 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e34903-5217-43ba-b007-c1102a571104 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.056276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e34903-5217-43ba-b007-c1102a571104 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.056552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e34903-5217-43ba-b007-c1102a571104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.070536 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e34903-5217-43ba-b007-c1102a571104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-35e34903-5217-43ba-b007-c1102a571104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:16.080918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:16.081959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:16.082846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:16.085185 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 391/797] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:16.086231 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e34903-5217-43ba-b007-c1102a571104 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.137709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b902ec-55a5-41b2-a84c-81396a5ec512 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.171668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33273db2-d671-4346-a628-c887140a1b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.172466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33273db2-d671-4346-a628-c887140a1b47 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.172748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33273db2-d671-4346-a628-c887140a1b47 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.173022 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33273db2-d671-4346-a628-c887140a1b47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.185363 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33273db2-d671-4346-a628-c887140a1b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-33273db2-d671-4346-a628-c887140a1b47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:16.193927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:16.194963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:16.195861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 407/798] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:16.196593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33273db2-d671-4346-a628-c887140a1b47 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.208130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e34903-5217-43ba-b007-c1102a571104 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.250518 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.251198 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.251489 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.251773 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.253221 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:16.253687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:16.255811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:16.279097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 392/799] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:16.279097 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.315606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33273db2-d671-4346-a628-c887140a1b47 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.350159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.351197 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.351702 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.352200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.353282 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:16.353848 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:16.355513 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:16.365295 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 408/800] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:16.366321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.382407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84c4d-3434-4974-9f35-fe752c2d29de None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.417320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.418001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.418290 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.418832 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.419316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:16.419625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:16.421470 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:16.433164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 393/801] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:16.433994 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.478497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce3919-6be0-4674-8da7-3910d05234cd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.527119 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.527788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.528074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.528352 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.529511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=17730dd371184663b2c0eb091cc668a3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:16.529817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:16.531595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:16.549080 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 409/802] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/role_assignments?user.id=17730dd371184663b2c0eb091cc668a3&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=58c34600086c469b87bfa39b74dd2607 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:16.549080 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.549946 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ea4b4-180f-40b3-86f0-b7d6d3439db7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.626932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.628159 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.628662 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.629261 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.652810 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:16.654678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:16.685424 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dc377-4ecc-4fbd-ab0c-dd4ec11295dd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:16.689267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:16.690281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:16.691317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:16.691317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:16.691317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:16.691317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:16.693640 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 394/803] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:16.701253 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.732991 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.735884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.736171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.736448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.759948 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:16.761454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:16.787969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:16.789020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:16.790016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:16.790016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:16.790016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:16.790016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:16.790719 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 410/804] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:16.798934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.859634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743ee3-d0fb-4c47-9d75-7d6d34280044 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.894289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.894940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.895229 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.895508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.896269 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:16.896583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:16.898031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:16.912239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 395/805] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:16.913753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497618c-b983-42a5-a5f2-4a5ea62fba62 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:16.918420 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:16.953126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:16.954135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:16.954613 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:16.955047 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:16.956487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:16.956951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:16.958536 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:16.972648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 411/806] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:16.977933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:17.020225 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d09780-9538-40e7-9fc4-0e302fb48fe1 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:17.055258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:17.056211 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.056621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.057003 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] PATH_INFO: `/v3/users/swift` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.064676 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:17.066878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:17.082539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:17.083803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:29:17.085128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:17.087069 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 396/807] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:17.093599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:17.095701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac46d36-5e83-4616-b226-6ac0a5f2f4cf None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:17.138146 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:17.139219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.139656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.140094 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.158215 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:17.159982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:17.181766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:17.182999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:17.184230 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:17.190965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 412/808] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:29:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:17.196760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:17.201383 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8ee1-6eb5-49ec-8fc7-9368d866e873 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:17.241540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:17.242402 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.242850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.243242 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.244285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:17.244710 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:17.246282 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:17.269169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 397/809] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:29:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:17.271316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:17.340007 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ddae2-9fd0-4b24-863e-00a601aec571 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:17.372379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44581e1e-0a2e-4aec-a77e-6bfd04165482 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:17.402275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:17.403730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.404444 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.405235 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.406488 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:17.407075 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:17.408925 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:17.409317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:17.409852 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.410211 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.410561 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.421295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:17.422845 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:17.441682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:17.442690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:17.443692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:17.443692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:17.443692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:17.443692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:17.443692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 398/810] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:17.448914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:17.451623 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 413/811] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:29:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:17.457357 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:17.550936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00584f-b9f5-4456-8588-2cd8dfa7c1f8 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:17.595411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:17.596121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.596403 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.596686 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.599693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:17.600013 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:17.601439 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:17.623787 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 399/812] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:17.632810 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:17.668613 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feccb02-96dd-4878-9ba3-5a483a0cfdd0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:17.740396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:17.741272 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.741554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.741829 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.759591 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:17.761066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:17.789582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:17.790603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:17.791570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:17.791570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:17.791570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:17.791570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:17.791570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 414/813] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:17.803027 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:17.818557 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae75e7b-cfde-4ac6-b804-64e8242aab12 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:17.861064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dd47c-e451-4933-b9d8-a5ec283ec8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:17.861064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dd47c-e451-4933-b9d8-a5ec283ec8e6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.861386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dd47c-e451-4933-b9d8-a5ec283ec8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.861627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dd47c-e451-4933-b9d8-a5ec283ec8e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.873771 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dd47c-e451-4933-b9d8-a5ec283ec8e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:17.877062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dd47c-e451-4933-b9d8-a5ec283ec8e6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-165dd47c-e451-4933-b9d8-a5ec283ec8e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:17.909354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:17.910744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:17.912753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:17.912753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:17.912753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:17.912753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:17.912753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:17.912753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:17.912753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:17.912753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:17.921130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 400/814] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:17.921130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d601-aaae-4dc7-b97f-446c29190820 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:17.921130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d601-aaae-4dc7-b97f-446c29190820 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:17.921130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d601-aaae-4dc7-b97f-446c29190820 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:17.921694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 401/815] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:17.922074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229d601-aaae-4dc7-b97f-446c29190820 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.088273 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6d673-ef80-49ab-bd73-68633ccc1dbe None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:18.100220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229d601-aaae-4dc7-b97f-446c29190820 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:18.156001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd0cd0-b98d-431b-a519-47d013e4a878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:18.157110 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd0cd0-b98d-431b-a519-47d013e4a878 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.157475 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd0cd0-b98d-431b-a519-47d013e4a878 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.157828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd0cd0-b98d-431b-a519-47d013e4a878 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.158685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd0cd0-b98d-431b-a519-47d013e4a878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:18.159230 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:18.159361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd0cd0-b98d-431b-a519-47d013e4a878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:18.159906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.160194 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.160508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.161519 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd0cd0-b98d-431b-a519-47d013e4a878 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:18.165076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:18.165076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:18.165076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:18.183991 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 415/816] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:18.185255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566675-a04c-4ad7-ac87-fb139f9bc806 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.185584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566675-a04c-4ad7-ac87-fb139f9bc806 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.185866 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566675-a04c-4ad7-ac87-fb139f9bc806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.189134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 402/817] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:18.190519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.218112 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-c7566675-a04c-4ad7-ac87-fb139f9bc806 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:18.219321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-c7566675-a04c-4ad7-ac87-fb139f9bc806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:18.309087 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 416/818] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:18.309087 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7566675-a04c-4ad7-ac87-fb139f9bc806 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.309577 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb18dd-7b5a-4e39-8470-192025cdabc2 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:18.354739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24420128-da8a-4d07-a9e1-66f10e8d79ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:18.355593 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24420128-da8a-4d07-a9e1-66f10e8d79ee None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.355879 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24420128-da8a-4d07-a9e1-66f10e8d79ee None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.356158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24420128-da8a-4d07-a9e1-66f10e8d79ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.365273 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24420128-da8a-4d07-a9e1-66f10e8d79ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:18.366699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24420128-da8a-4d07-a9e1-66f10e8d79ee None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-24420128-da8a-4d07-a9e1-66f10e8d79ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:18.386517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:18.387583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:18.388542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 403/819] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:18.389567 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae1bb-3174-466d-9231-497a2517dd79 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.389676 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae1bb-3174-466d-9231-497a2517dd79 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.389982 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ae1bb-3174-466d-9231-497a2517dd79 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.391317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 404/820] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:18.393805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ae1bb-3174-466d-9231-497a2517dd79 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.427792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7566675-a04c-4ad7-ac87-fb139f9bc806 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:18.461423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5324c4a-88d1-41bb-8c43-a4093d96288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:18.462233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5324c4a-88d1-41bb-8c43-a4093d96288c None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.462521 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5324c4a-88d1-41bb-8c43-a4093d96288c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.462871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5324c4a-88d1-41bb-8c43-a4093d96288c None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/a752b7105b754628a153e396a9110653/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.464863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5324c4a-88d1-41bb-8c43-a4093d96288c None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=a752b7105b754628a153e396a9110653, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:18.466354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5324c4a-88d1-41bb-8c43-a4093d96288c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:18.489144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 417/821] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:29:18 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/a752b7105b754628a153e396a9110653/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:29:18.490679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5324c4a-88d1-41bb-8c43-a4093d96288c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.504297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ae1bb-3174-466d-9231-497a2517dd79 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:18.541380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af02ea-aac7-46d7-a931-c7a448ff3044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:18.542222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af02ea-aac7-46d7-a931-c7a448ff3044 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.542508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af02ea-aac7-46d7-a931-c7a448ff3044 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.542790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af02ea-aac7-46d7-a931-c7a448ff3044 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.543572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af02ea-aac7-46d7-a931-c7a448ff3044 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:18.545012 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af02ea-aac7-46d7-a931-c7a448ff3044 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-25af02ea-aac7-46d7-a931-c7a448ff3044 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:18.558958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:18.560126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:18.561031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 405/822] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:29:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:18.561982 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9467cd-f941-4c19-9b66-6937455be207 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.562666 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9467cd-f941-4c19-9b66-6937455be207 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.562666 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9467cd-f941-4c19-9b66-6937455be207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.580686 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-6c9467cd-f941-4c19-9b66-6937455be207 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:18.581993 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-6c9467cd-f941-4c19-9b66-6937455be207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:18.616648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5324c4a-88d1-41bb-8c43-a4093d96288c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:18.654401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:18.655613 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.656114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.656621 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.657685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:18.658236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:18.660452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:18.661448 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 406/823] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:18.662478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9467cd-f941-4c19-9b66-6937455be207 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.674119 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 418/824] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:18.680986 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.778996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9467cd-f941-4c19-9b66-6937455be207 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:18.807389 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb958b66-1f65-421b-a88a-480cc8a40da0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:18.815941 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:18.816621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.816911 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.817236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.817964 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:18.818277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:18.819667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:18.824497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 407/825] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:29:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:18.829596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.847378 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65573d-501f-4b5e-9a34-5bdfbf2c6830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:18.848447 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65573d-501f-4b5e-9a34-5bdfbf2c6830 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:18.848730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65573d-501f-4b5e-9a34-5bdfbf2c6830 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:18.849317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65573d-501f-4b5e-9a34-5bdfbf2c6830 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/8db5070e21174e5593f244bdb2c52224/roles/582e6b7be6a445019c4cd90d2dcd594a` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:18.869241 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65573d-501f-4b5e-9a34-5bdfbf2c6830 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=8db5070e21174e5593f244bdb2c52224, role_id=582e6b7be6a445019c4cd90d2dcd594a)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:18.870707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65573d-501f-4b5e-9a34-5bdfbf2c6830 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:18.886661 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 419/826] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:29:18 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/8db5070e21174e5593f244bdb2c52224/roles/582e6b7be6a445019c4cd90d2dcd594a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:29:18.889960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65573d-501f-4b5e-9a34-5bdfbf2c6830 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:18.985918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdec6b0-6216-4d46-a0cf-9ee77d2fc152 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:19.030234 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:19.031029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.031498 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.031825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] PATH_INFO: `/v3/services/network` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.032506 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:19.034504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:19.047301 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:19.049859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:19.051937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 408/827] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:18 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:19.061066 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:19.088338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65573d-501f-4b5e-9a34-5bdfbf2c6830 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:19.164031 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f833c3-4b80-4d7c-b3f7-3e390d49632d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:19.185507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:19.186203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.186490 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.186771 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.187503 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:19.187819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:19.189246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:19.201920 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:19.202670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.202966 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.203307 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.204043 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] RBAC: Adding query filter params (name=network) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:19.204363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:19.206468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:19.208165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 420/828] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:29:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:19.211891 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 409/829] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:19.215637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:19.217326 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:19.389602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fb58b-ec27-474a-8976-c54e6857be42 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:19.399801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef1e87-ad2c-44b1-9ac9-56a650495c86 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:19.450885 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438fd04-79c7-4d92-b5ff-1d7d72115ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:19.451555 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438fd04-79c7-4d92-b5ff-1d7d72115ba7 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.451837 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438fd04-79c7-4d92-b5ff-1d7d72115ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.452113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438fd04-79c7-4d92-b5ff-1d7d72115ba7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.452869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438fd04-79c7-4d92-b5ff-1d7d72115ba7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:19.454276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438fd04-79c7-4d92-b5ff-1d7d72115ba7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:19.465368 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:19.466038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.466324 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.466602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.467325 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] RBAC: Adding query filter params (type=network) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:19.467631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:19.468995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:19.483658 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 410/830] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:19.490419 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:19.514455 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 421/831] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:29:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:19.550292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062200-a85d-46c6-b30d-f223628909e9 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.550638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062200-a85d-46c6-b30d-f223628909e9 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.550905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062200-a85d-46c6-b30d-f223628909e9 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.552717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 422/832] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:19.558883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d36b10-ad31-4090-8992-d108b36a0126 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.559357 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d36b10-ad31-4090-8992-d108b36a0126 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.559750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d36b10-ad31-4090-8992-d108b36a0126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.578819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-62d36b10-ad31-4090-8992-d108b36a0126 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:19.580233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-62d36b10-ad31-4090-8992-d108b36a0126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:19.628436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3c4dc-8537-40b1-83cb-2dcefd279c75 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:19.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbdc82-bc0d-4773-bef0-2570cc1e0e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:19.676425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbdc82-bc0d-4773-bef0-2570cc1e0e15 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.676716 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbdc82-bc0d-4773-bef0-2570cc1e0e15 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.677001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbdc82-bc0d-4773-bef0-2570cc1e0e15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.677988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbdc82-bc0d-4773-bef0-2570cc1e0e15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:19.679407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbdc82-bc0d-4773-bef0-2570cc1e0e15 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:19.710446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 411/833] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:29:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:19.771690 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 423/834] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:19.795641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbdc82-bc0d-4773-bef0-2570cc1e0e15 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:19.913173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbdc82-bc0d-4773-bef0-2570cc1e0e15 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:19.949096 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:19.949856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:19.950148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:19.950434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:19.951118 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:19.952498 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:19.966170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:19.967232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:19.968130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 412/835] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:29:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:19.973969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d36b10-ad31-4090-8992-d108b36a0126 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:20.081009 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d36b10-ad31-4090-8992-d108b36a0126 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:20.123338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95acf07f-0782-431d-a42a-c8d2900e1212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:20.124626 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acf07f-0782-431d-a42a-c8d2900e1212 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:20.124921 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acf07f-0782-431d-a42a-c8d2900e1212 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:20.125232 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95acf07f-0782-431d-a42a-c8d2900e1212 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:20.125967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acf07f-0782-431d-a42a-c8d2900e1212 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:20.126280 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acf07f-0782-431d-a42a-c8d2900e1212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:20.127729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95acf07f-0782-431d-a42a-c8d2900e1212 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:20.132810 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 424/836] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:20.138126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:20.243687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5455737-068c-434a-8158-f5c470af6574 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:20.283393 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c646148-0b6e-4ac2-b09c-3c0f787a401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:20.284628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646148-0b6e-4ac2-b09c-3c0f787a401d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:20.284919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646148-0b6e-4ac2-b09c-3c0f787a401d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:20.285201 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c646148-0b6e-4ac2-b09c-3c0f787a401d None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:20.285962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c646148-0b6e-4ac2-b09c-3c0f787a401d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:20.286274 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c646148-0b6e-4ac2-b09c-3c0f787a401d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:20.287650 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c646148-0b6e-4ac2-b09c-3c0f787a401d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:20.292428 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 413/837] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:20 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:20.891025 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78125-bb0e-4417-8364-abfbea853b1a None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:20.891736 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78125-bb0e-4417-8364-abfbea853b1a None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:20.892264 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd78125-bb0e-4417-8364-abfbea853b1a None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:20.894186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 425/838] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:20.899725 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e789-2439-4155-bd4c-0c06d3cfce0b None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:20.900584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e789-2439-4155-bd4c-0c06d3cfce0b None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:20.901116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6e789-2439-4155-bd4c-0c06d3cfce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:20.919048 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-dfe6e789-2439-4155-bd4c-0c06d3cfce0b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:20.920590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-dfe6e789-2439-4155-bd4c-0c06d3cfce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:20.992194 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 414/839] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:21.015966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd78125-bb0e-4417-8364-abfbea853b1a None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:21.133668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd78125-bb0e-4417-8364-abfbea853b1a None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:21.169160 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8ca5d-065f-4e44-83c7-f7f74309d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:21.170327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ca5d-065f-4e44-83c7-f7f74309d2a0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:21.170850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ca5d-065f-4e44-83c7-f7f74309d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:21.171339 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ca5d-065f-4e44-83c7-f7f74309d2a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:21.182159 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8ca5d-065f-4e44-83c7-f7f74309d2a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-61a8ca5d-065f-4e44-83c7-f7f74309d2a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:21.190569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:21.191570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:21.192524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:21.195343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 426/840] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:21.201477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6e789-2439-4155-bd4c-0c06d3cfce0b None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:21.305986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6e789-2439-4155-bd4c-0c06d3cfce0b None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:21.339501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563f33a-9ab0-4cdf-a29b-fef93c921b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:21.340202 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563f33a-9ab0-4cdf-a29b-fef93c921b2a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:21.340486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563f33a-9ab0-4cdf-a29b-fef93c921b2a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:21.340764 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563f33a-9ab0-4cdf-a29b-fef93c921b2a None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:21.341574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563f33a-9ab0-4cdf-a29b-fef93c921b2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:21.341889 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563f33a-9ab0-4cdf-a29b-fef93c921b2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:21.343312 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563f33a-9ab0-4cdf-a29b-fef93c921b2a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:21.353977 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 415/841] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:21.360072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8ca5d-065f-4e44-83c7-f7f74309d2a0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:21.466948 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8ca5d-065f-4e44-83c7-f7f74309d2a0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:21.502491 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:21.503680 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:21.504181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:21.504672 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:21.516190 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:21.518505 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:21.536704 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:21.537746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:21.538710 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:21.538710 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:21.538710 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:21.538710 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:21.541026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 427/842] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:21.546716 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e215b-dd44-4b41-9625-5c67bae77126 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:21.546867 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:21.547046 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e215b-dd44-4b41-9625-5c67bae77126 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:21.547413 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e215b-dd44-4b41-9625-5c67bae77126 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:21.548891 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 416/843] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:21.558068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764caaf-dffd-41d0-90fb-1b63a9a3e4e9 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:21.559012 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764caaf-dffd-41d0-90fb-1b63a9a3e4e9 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:21.559516 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764caaf-dffd-41d0-90fb-1b63a9a3e4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:21.582378 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-6764caaf-dffd-41d0-90fb-1b63a9a3e4e9 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:21.583586 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-6764caaf-dffd-41d0-90fb-1b63a9a3e4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:21.665064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 417/844] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:21.666869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dae5e-ee0c-4ea0-b48c-7be7cd5ae9c4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:21.683487 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764caaf-dffd-41d0-90fb-1b63a9a3e4e9 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:21.702396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:21.703087 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:21.703433 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:21.703738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:21.704468 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:21.704785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:21.706914 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:21.724329 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 428/845] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:21.725371 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:21.795234 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764caaf-dffd-41d0-90fb-1b63a9a3e4e9 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:21.828694 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffdbbd-9ee8-47e4-807e-3f795d9dbdd3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:21.831657 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e009-27c1-4024-85de-e6df00709e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:21.832729 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e009-27c1-4024-85de-e6df00709e5a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:21.833281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e009-27c1-4024-85de-e6df00709e5a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:21.833776 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46e009-27c1-4024-85de-e6df00709e5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:21.844604 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46e009-27c1-4024-85de-e6df00709e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-6f46e009-27c1-4024-85de-e6df00709e5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:21.852763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:21.853818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:29:21.854757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:21.861269 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 418/846] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:29:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:21.863922 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e009-27c1-4024-85de-e6df00709e5a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:21.864911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:21.865749 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:21.866038 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:21.866320 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] PATH_INFO: `/v3/users/swift` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:21.874382 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:21.875566 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:21.891291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:21.892332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:29:21.894345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:21.898126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 429/847] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:21.898923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:21.976650 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46e009-27c1-4024-85de-e6df00709e5a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.013338 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.014045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.014334 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.014619 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.015386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:22.015790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.017130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.029417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 419/848] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:29:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:22.033300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.039516 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc9a4a-3c39-47e1-8341-c4f7bf629036 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.084593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.085722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.086229 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.086722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.087928 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:22.088473 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.090167 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.117126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 430/849] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:29:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:22.119118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.155667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9118d-a26b-4bab-b49c-9e11e896ac98 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.189432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424a19d-81fa-49c4-9435-4419cf2912a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.190371 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424a19d-81fa-49c4-9435-4419cf2912a7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.190738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424a19d-81fa-49c4-9435-4419cf2912a7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.191025 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424a19d-81fa-49c4-9435-4419cf2912a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.201728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424a19d-81fa-49c4-9435-4419cf2912a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.203186 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424a19d-81fa-49c4-9435-4419cf2912a7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-c424a19d-81fa-49c4-9435-4419cf2912a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:22.220456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:22.222103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:22.223059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:22.223059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:22.223059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:22.223059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:22.223059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 420/850] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:22.224053 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4eaa8-89bd-4447-b550-07655fc58eb3 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.224279 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4eaa8-89bd-4447-b550-07655fc58eb3 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.224557 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4eaa8-89bd-4447-b550-07655fc58eb3 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.225949 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 421/851] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:22.230801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4eaa8-89bd-4447-b550-07655fc58eb3 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.235074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af87f26-4e4f-4ebc-ae3c-3b4559f95d7e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.271423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef577-be66-45dd-9220-cc096749a280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.272074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef577-be66-45dd-9220-cc096749a280 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.272701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef577-be66-45dd-9220-cc096749a280 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.272987 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef577-be66-45dd-9220-cc096749a280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.329122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4eaa8-89bd-4447-b550-07655fc58eb3 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.341590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef577-be66-45dd-9220-cc096749a280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.343183 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef577-be66-45dd-9220-cc096749a280 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.363754 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5cd82-c63d-497f-8625-cbd3faafece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.364439 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5cd82-c63d-497f-8625-cbd3faafece3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.364733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5cd82-c63d-497f-8625-cbd3faafece3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.365019 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5cd82-c63d-497f-8625-cbd3faafece3 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.365787 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5cd82-c63d-497f-8625-cbd3faafece3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:22.366099 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5cd82-c63d-497f-8625-cbd3faafece3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.367486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5cd82-c63d-497f-8625-cbd3faafece3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.380237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 422/852] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:22.381662 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64af963-0b68-4388-9946-385c0c7a4323 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.381959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64af963-0b68-4388-9946-385c0c7a4323 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.382245 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64af963-0b68-4388-9946-385c0c7a4323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.401412 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-b64af963-0b68-4388-9946-385c0c7a4323 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:22.402660 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-b64af963-0b68-4388-9946-385c0c7a4323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:22.448143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 431/853] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:29:22.453382 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef577-be66-45dd-9220-cc096749a280 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.471921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 423/854] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:22.473130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64af963-0b68-4388-9946-385c0c7a4323 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.560720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef577-be66-45dd-9220-cc096749a280 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.572306 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64af963-0b68-4388-9946-385c0c7a4323 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.593579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.594386 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.594671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.594944 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.601782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.603218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.605345 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922273f-18cb-4251-977b-c47878633ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.606136 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922273f-18cb-4251-977b-c47878633ada None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.606421 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922273f-18cb-4251-977b-c47878633ada None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.606701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922273f-18cb-4251-977b-c47878633ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.616759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922273f-18cb-4251-977b-c47878633ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:22.618055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:22.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:22.620500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:22.621222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922273f-18cb-4251-977b-c47878633ada None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.621310 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 432/855] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:22.621310 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-1922273f-18cb-4251-977b-c47878633ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:22.639928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:22.641147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:22.642131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:22.642131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:22.642131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:22.642131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:22.642131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 424/856] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:22.642430 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859fa6f-08a4-43af-87ea-cf6efaa8068a None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.642635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859fa6f-08a4-43af-87ea-cf6efaa8068a None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.642973 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859fa6f-08a4-43af-87ea-cf6efaa8068a None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.644743 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 425/857] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:22.649484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859fa6f-08a4-43af-87ea-cf6efaa8068a None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.716759 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bddb00-7499-4e42-bf2b-8b9bc582a225 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.745451 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859fa6f-08a4-43af-87ea-cf6efaa8068a None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.750501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.751298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.751582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.751861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.753134 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.754515 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:22.766148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:22.767203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:22.768135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 433/858] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:22.769350 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.783552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826231a-2abc-46f5-a080-982f833ee06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.784222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826231a-2abc-46f5-a080-982f833ee06e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.784505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826231a-2abc-46f5-a080-982f833ee06e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.784786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826231a-2abc-46f5-a080-982f833ee06e None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.785675 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826231a-2abc-46f5-a080-982f833ee06e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:22.785991 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826231a-2abc-46f5-a080-982f833ee06e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.787417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826231a-2abc-46f5-a080-982f833ee06e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.804667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 426/859] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:22.805917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb16ae-4e00-4663-8ed8-b3b5ef6746dc None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.806215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb16ae-4e00-4663-8ed8-b3b5ef6746dc None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.806497 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb16ae-4e00-4663-8ed8-b3b5ef6746dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.822496 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-e4fb16ae-4e00-4663-8ed8-b3b5ef6746dc None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:22.823620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-e4fb16ae-4e00-4663-8ed8-b3b5ef6746dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:22.863017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3ff93-534c-4dae-b938-4936744497a7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:22.886360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 427/860] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:22.887180 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb16ae-4e00-4663-8ed8-b3b5ef6746dc None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.894765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:22.895418 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:22.895701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:22.895974 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:22.896692 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:22.896999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:22.898417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:22.910132 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 434/861] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:22.910952 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:22.980096 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb16ae-4e00-4663-8ed8-b3b5ef6746dc None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.005156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745fe05-7362-4406-9c36-c9bb1152f183 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.013856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.014513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.014793 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.015066 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.015785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:23.016089 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.017485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.021971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 428/862] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:23.022712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:23.043258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a3826-1b7c-4588-bf06-33b4f60344e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.043910 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a3826-1b7c-4588-bf06-33b4f60344e0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.044194 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a3826-1b7c-4588-bf06-33b4f60344e0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.044473 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a3826-1b7c-4588-bf06-33b4f60344e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.112379 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a3826-1b7c-4588-bf06-33b4f60344e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.113871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a3826-1b7c-4588-bf06-33b4f60344e0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.123468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5668c40-a64a-4210-b65d-f9a22ed23044 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.155678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af603d42-dd0f-476d-854a-5e13a7e69ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.156348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af603d42-dd0f-476d-854a-5e13a7e69ae8 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.156653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af603d42-dd0f-476d-854a-5e13a7e69ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.156913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af603d42-dd0f-476d-854a-5e13a7e69ae8 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.157670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af603d42-dd0f-476d-854a-5e13a7e69ae8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.159572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af603d42-dd0f-476d-854a-5e13a7e69ae8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.172092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 429/863] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:29:23 2022] POST /identity/v3/services => generated 269 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:23.172881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af603d42-dd0f-476d-854a-5e13a7e69ae8 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:23.226161 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 435/864] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:23.232445 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a3826-1b7c-4588-bf06-33b4f60344e0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:23.289663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af603d42-dd0f-476d-854a-5e13a7e69ae8 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.324477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.325333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.325618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.325895 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.333210 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.334685 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.334914 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a3826-1b7c-4588-bf06-33b4f60344e0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:23.351004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:23.352961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 430/865] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:23.354199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:23.373922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.374562 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.374846 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.375119 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.375832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:23.376144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.377603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.389068 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 436/866] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:23.389068 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:23.487746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24319f1a-eb59-4d4e-94b0-0a03edfa23ff None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.502662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210699c-d3d9-47d7-92c4-d569e4fd5c8f None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.534615 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.535482 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.535782 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.536084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.542637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.542920 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.543767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.544589 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:23.545336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.547107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.547237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.548559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.561919 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 437/867] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:23.567386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:23.568375 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:23.570057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:23.570057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:23.570057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:23.570057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:23.570607 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:23.570774 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 431/868] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:23.578053 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:23.677725 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b466a-2ef0-4380-8e2d-e71712a84b57 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.687212 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1366d-cb14-42df-881f-e1453b69e91a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.711880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.712537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.712818 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.713118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.713961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=a752b7105b754628a153e396a9110653) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:23.714270 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.715707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.722732 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 438/869] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:23 2022] GET /identity/v3/role_assignments?user.id=a752b7105b754628a153e396a9110653&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=58c34600086c469b87bfa39b74dd2607 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:23.730047 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.730846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.731200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.731545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.732659 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:23.733067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.734596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.747999 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 432/870] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:23.753214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:23.864811 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ef4af-1cb6-4145-a7e2-601337d05bd0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:23.903699 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:23.904512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:23.904784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:23.905072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:23.912790 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:23.914287 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:23.929422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:23.931614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:23.933791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 439/871] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:23.936819 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:24.039233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697364e-5849-4407-8247-9b984330e5a8 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:24.075184 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d706214-30db-4473-b64b-38d8cfb56567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:24.076170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d706214-30db-4473-b64b-38d8cfb56567 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:24.076642 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d706214-30db-4473-b64b-38d8cfb56567 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:24.077169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d706214-30db-4473-b64b-38d8cfb56567 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:24.078151 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d706214-30db-4473-b64b-38d8cfb56567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:24.078662 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d706214-30db-4473-b64b-38d8cfb56567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:24.080291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d706214-30db-4473-b64b-38d8cfb56567 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:24.094082 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 433/872] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:24.105064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:24.202129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97311940-ddb4-48c3-9b51-12b2c2e70188 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:24.237366 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890e09c-b56a-480c-aaec-35126ecea934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:24.238026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890e09c-b56a-480c-aaec-35126ecea934 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:24.238304 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890e09c-b56a-480c-aaec-35126ecea934 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:24.238579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890e09c-b56a-480c-aaec-35126ecea934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:24.239421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890e09c-b56a-480c-aaec-35126ecea934 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=8db5070e21174e5593f244bdb2c52224) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:24.239736 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890e09c-b56a-480c-aaec-35126ecea934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:24.241235 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890e09c-b56a-480c-aaec-35126ecea934 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:24.247587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 440/873] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:24 2022] GET /identity/v3/role_assignments?user.id=8db5070e21174e5593f244bdb2c52224&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:25.551877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754882-0687-4881-85a3-3141af6a958d None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:25.552413 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754882-0687-4881-85a3-3141af6a958d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:25.552413 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754882-0687-4881-85a3-3141af6a958d None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:25.554112 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 434/874] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:25.559882 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce85587-382e-4c59-aad7-b655d2c7899b None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:25.560422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce85587-382e-4c59-aad7-b655d2c7899b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:25.560422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce85587-382e-4c59-aad7-b655d2c7899b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:25.579548 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-9ce85587-382e-4c59-aad7-b655d2c7899b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:25.581440 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-9ce85587-382e-4c59-aad7-b655d2c7899b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:25.652516 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 441/875] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:25 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:25.674934 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13754882-0687-4881-85a3-3141af6a958d None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:25.765763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701580-dc24-453b-ace2-f537bac1c154 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:25.766101 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701580-dc24-453b-ace2-f537bac1c154 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:25.766326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701580-dc24-453b-ace2-f537bac1c154 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:25.767999 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 442/876] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:25.774270 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d123-7201-42d0-86ce-33c91399b61a None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:25.774666 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d123-7201-42d0-86ce-33c91399b61a None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:25.775053 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d123-7201-42d0-86ce-33c91399b61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:25.782690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13754882-0687-4881-85a3-3141af6a958d None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:25.791892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-0871d123-7201-42d0-86ce-33c91399b61a None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:25.793259 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-0871d123-7201-42d0-86ce-33c91399b61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:25.817926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:25.818675 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:25.818959 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:25.819237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:25.819896 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:25.821292 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:25.833831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:25.835407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:25.836634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 435/877] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:29:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:25.841003 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:25.869366 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 443/878] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:25 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:25.892479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871d123-7201-42d0-86ce-33c91399b61a None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:25.947771 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d18868-63f0-4ec5-b4fe-ded3246dc640 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:25.981672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:25.982614 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:25.982912 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:25.983256 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:25.983977 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:25.984344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:25.985738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:25.991272 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 436/879] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:25.996791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:25.998102 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871d123-7201-42d0-86ce-33c91399b61a None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:26.035040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647b298-3273-428a-b8bf-d40cb4508f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:26.035823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b298-3273-428a-b8bf-d40cb4508f49 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.036227 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b298-3273-428a-b8bf-d40cb4508f49 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.036578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647b298-3273-428a-b8bf-d40cb4508f49 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.037434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647b298-3273-428a-b8bf-d40cb4508f49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:26.039383 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647b298-3273-428a-b8bf-d40cb4508f49 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:26.058791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 444/880] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:29:25 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:29:26.117289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5c7f8-d562-4496-9da9-60b87ef4500b None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.117289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5c7f8-d562-4496-9da9-60b87ef4500b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.118546 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5c7f8-d562-4496-9da9-60b87ef4500b None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.122527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 445/881] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:26.126110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67364e-ed6d-432b-8c56-49ff19bbcd3d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:26.126693 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558a2a8-ed93-47f0-b525-b967b00ad0bd None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.126988 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558a2a8-ed93-47f0-b525-b967b00ad0bd None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.127464 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558a2a8-ed93-47f0-b525-b967b00ad0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.146769 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-4558a2a8-ed93-47f0-b525-b967b00ad0bd None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:26.148010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-4558a2a8-ed93-47f0-b525-b967b00ad0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:26.171035 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:26.172160 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.172703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.173125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.173989 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:26.174993 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:26.176549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:26.182370 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 437/882] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:26.193093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:26.240226 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 446/883] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:26.262053 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558a2a8-ed93-47f0-b525-b967b00ad0bd None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:26.333545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093cb7d-06de-4803-a7fd-9cb9d1e1e5ed None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:26.369882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:26.370943 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.371502 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.372005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] PATH_INFO: `/v3/endpoints` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.373104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] RBAC: Adding query filter params (interface=public, service_id=5adfddb6c1984167862e5ba7ce45293e, region_id=RegionOne) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:26.373650 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:26.375244 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:26.381136 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 438/884] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:29:26 2022] GET /identity/v3/endpoints?service_id=5adfddb6c1984167862e5ba7ce45293e&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:26.386980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:26.402700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558a2a8-ed93-47f0-b525-b967b00ad0bd None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:26.450090 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:26.451502 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.452126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.452940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.454079 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:26.456232 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:26.472496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:26.474074 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:26.475277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:26.478752 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 447/885] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:26.484778 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:26.507522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e79c17-8aeb-4c16-a355-02ec4e81c32f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:26.542311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4b322-c285-4969-8c12-d9888bbf8ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:26.543397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b322-c285-4969-8c12-d9888bbf8ea9 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.543906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b322-c285-4969-8c12-d9888bbf8ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.544409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b322-c285-4969-8c12-d9888bbf8ea9 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.545346 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4b322-c285-4969-8c12-d9888bbf8ea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:26.546924 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4b322-c285-4969-8c12-d9888bbf8ea9 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:26.553170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 439/886] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:29:26 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:29:26.581622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a64d-3e39-4dd7-8137-beccd9e87b71 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.582316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a64d-3e39-4dd7-8137-beccd9e87b71 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.582316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201a64d-3e39-4dd7-8137-beccd9e87b71 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.588029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 440/887] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:26.589648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a988ef6-45ed-463c-ac43-0b8c6f448e8b None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.589855 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a988ef6-45ed-463c-ac43-0b8c6f448e8b None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.590100 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a988ef6-45ed-463c-ac43-0b8c6f448e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.601617 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6736bf-9485-4bb7-b43d-f9b27895ebb6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:26.608477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-7a988ef6-45ed-463c-ac43-0b8c6f448e8b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:26.609730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-7a988ef6-45ed-463c-ac43-0b8c6f448e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:26.640318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:26.641406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.641937 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.642429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.643939 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:26.644596 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:26.646439 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:26.656935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 448/888] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:26.658315 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:26.689134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 441/889] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:26.708105 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a988ef6-45ed-463c-ac43-0b8c6f448e8b None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:26.794303 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee248aa3-8372-4cb0-bade-65901a403093 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:26.829985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:26.831216 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.831504 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.831782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.831897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a988ef6-45ed-463c-ac43-0b8c6f448e8b None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:26.832570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:26.832883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:26.834360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:26.839279 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 449/890] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:26.872195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9dbff-4625-4470-bc06-32c22d93fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:26.873190 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9dbff-4625-4470-bc06-32c22d93fb61 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:26.873480 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9dbff-4625-4470-bc06-32c22d93fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:26.873762 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9dbff-4625-4470-bc06-32c22d93fb61 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:26.886629 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9dbff-4625-4470-bc06-32c22d93fb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-adb9dbff-4625-4470-bc06-32c22d93fb61 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:26.895942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:26.897028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:26.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 442/891] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:29:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:26.905475 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:27.014305 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be8395-1ef0-48b1-93e4-74c255cbd590 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:27.050748 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:27.051426 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:27.051756 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:27.052039 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:27.052840 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:27.053182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:27.054618 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:27.063165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 450/892] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:29:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:27.068605 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9dbff-4625-4470-bc06-32c22d93fb61 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:27.181926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9dbff-4625-4470-bc06-32c22d93fb61 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:27.219635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:27.220441 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:27.220728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:27.221589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:27.233010 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:27.234495 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:27.254121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:27.255143 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:27.256104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:27.256104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:27.256104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:27.256104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:27.256104 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 443/893] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:27.260907 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:27.363906 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0730e6-7828-43e9-b514-8bccf979e1f6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:27.398578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:27.399253 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:27.399538 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:27.399813 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:27.400550 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:27.400859 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:27.402275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:27.415298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 451/894] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:27.420435 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:27.526926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b256a8b-7646-454e-aea0-719311cdcdd3 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:27.561634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:27.562447 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:27.562735 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:27.563011 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:27.571126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:27.572579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:27.587540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:27.588571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:27.589697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 444/895] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:29:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:27.594564 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:27.694796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd6434-a9b1-4653-9fe8-8a2ba60c3781 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:27.728992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:27.729695 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:27.729975 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:27.730247 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:27.731097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:27.731403 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:27.733118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:27.750873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 452/896] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:29:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:27.756962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:27.896888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8837044-707c-4671-9026-beaa3b3335b5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:27.931017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b298ed-927e-47c9-808a-1d040cd18a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:27.931714 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b298ed-927e-47c9-808a-1d040cd18a56 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:27.931975 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b298ed-927e-47c9-808a-1d040cd18a56 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:27.932258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b298ed-927e-47c9-808a-1d040cd18a56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.004515 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b298ed-927e-47c9-808a-1d040cd18a56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:28.006525 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b298ed-927e-47c9-808a-1d040cd18a56 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:28.109694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 445/897] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:27 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:28.115938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:28.235522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734570e-37c8-4365-88fc-6d0e6fb34b45 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:28.274747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee8523-a51e-4379-bfd6-f9b759d8eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:28.275554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8523-a51e-4379-bfd6-f9b759d8eeee None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.275892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8523-a51e-4379-bfd6-f9b759d8eeee None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.276177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8523-a51e-4379-bfd6-f9b759d8eeee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.286755 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee8523-a51e-4379-bfd6-f9b759d8eeee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:28.288223 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee8523-a51e-4379-bfd6-f9b759d8eeee None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-44ee8523-a51e-4379-bfd6-f9b759d8eeee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:28.306276 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:28.307274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:28.308634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:28.308634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:28.308634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:28.308634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:28.308634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 453/898] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:28.313409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b298ed-927e-47c9-808a-1d040cd18a56 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:28.414197 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b298ed-927e-47c9-808a-1d040cd18a56 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:28.433318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffa7b3-d7a6-4727-95f8-51efc74e6b0b None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.433646 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffa7b3-d7a6-4727-95f8-51efc74e6b0b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.433883 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffa7b3-d7a6-4727-95f8-51efc74e6b0b None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.435519 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 454/899] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:28.440888 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb1a88-ed08-47eb-b644-6765e7d9e1d7 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.441219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb1a88-ed08-47eb-b644-6765e7d9e1d7 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.441501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb1a88-ed08-47eb-b644-6765e7d9e1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.450119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:28.450792 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.451080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.451363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.452092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:28.452836 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:28.454544 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:28.458336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-5feb1a88-ed08-47eb-b644-6765e7d9e1d7 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:28.459522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-5feb1a88-ed08-47eb-b644-6765e7d9e1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:28.467849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 446/900] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:28.474254 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:28.529257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 455/901] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:28.552420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb1a88-ed08-47eb-b644-6765e7d9e1d7 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:28.578910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a001d0e-e0e7-4a78-825f-3ffbd5bdfd0a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:28.614074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:28.614880 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.615169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.615448 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.622276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:28.623734 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:28.638736 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:28.641057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:28.643499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 447/902] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:28.647803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:28.651729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb1a88-ed08-47eb-b644-6765e7d9e1d7 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:28.688347 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:28.689175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.689455 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.689763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.690426 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:28.691816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:28.703654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:28.707186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:28.709471 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:28.718101 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 456/903] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:28.718101 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:28.748701 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef114109-09bb-40e9-ba2d-d2cababfb566 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:28.787016 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:28.787611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.787896 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.788177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.788981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:28.789316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:28.790775 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:28.807832 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 448/904] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:28.808912 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:28.816870 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf616ef-4fac-4c58-b1dd-92493e5ee4ff None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:28.851765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:28.852438 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.852724 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.853003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.853762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:28.854073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:28.855452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:28.865370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 457/905] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:28.866363 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:28.911054 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd34c4-9773-4413-863b-150f703189d2 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:28.945390 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bb335-bdd6-4474-96d6-c7cb1357bf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:28.946063 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bb335-bdd6-4474-96d6-c7cb1357bf9b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.946350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bb335-bdd6-4474-96d6-c7cb1357bf9b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.946667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bb335-bdd6-4474-96d6-c7cb1357bf9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.947466 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bb335-bdd6-4474-96d6-c7cb1357bf9b None admin] RBAC: Adding query filter params (role.id=87f96af882934745afa3b46c6d539918, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=8db5070e21174e5593f244bdb2c52224) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:28.947791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bb335-bdd6-4474-96d6-c7cb1357bf9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:28.949253 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bb335-bdd6-4474-96d6-c7cb1357bf9b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:28.957609 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 449/906] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:28 2022] GET /identity/v3/role_assignments?user.id=8db5070e21174e5593f244bdb2c52224&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=87f96af882934745afa3b46c6d539918 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:28.959092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9f04e-ab4b-4505-9ffb-bf01a64d89e4 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.959382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9f04e-ab4b-4505-9ffb-bf01a64d89e4 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.959706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9f04e-ab4b-4505-9ffb-bf01a64d89e4 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.961250 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 450/907] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:28.966699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbdac-2ff9-4ba1-a656-e6da933de5ad None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:28.967006 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbdac-2ff9-4ba1-a656-e6da933de5ad None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:28.967267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbdac-2ff9-4ba1-a656-e6da933de5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:28.978479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52daef6-76ad-48f6-bf26-a27f48f3d518 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:28.986344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-980fbdac-2ff9-4ba1-a656-e6da933de5ad None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:28.987671 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-980fbdac-2ff9-4ba1-a656-e6da933de5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:29.025132 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:29.027387 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.027951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.028434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.029410 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:29.029943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:29.031559 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:29.044667 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 458/908] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:29.046330 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:29.077486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 451/909] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:29.106125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fbdac-2ff9-4ba1-a656-e6da933de5ad None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:29.183375 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76365a7-a03c-451c-9e78-0dcda66f092a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:29.220169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fbdac-2ff9-4ba1-a656-e6da933de5ad None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:29.226903 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:29.227571 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.227857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.228135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.228939 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] RBAC: Adding query filter params (interface=public, service_id=388dffa837e54ec28a1ca699abea4d3b, region_id=RegionOne) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:29.229297 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:29.230924 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:29.240662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 459/910] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:29:29 2022] GET /identity/v3/endpoints?service_id=388dffa837e54ec28a1ca699abea4d3b&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:29.243174 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:29.267644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd7cc0-3da5-44e1-9202-f99d5836efd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:29.269024 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd7cc0-3da5-44e1-9202-f99d5836efd0 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.269333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd7cc0-3da5-44e1-9202-f99d5836efd0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.269582 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd7cc0-3da5-44e1-9202-f99d5836efd0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.270250 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd7cc0-3da5-44e1-9202-f99d5836efd0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:29.271623 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd7cc0-3da5-44e1-9202-f99d5836efd0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-80cd7cc0-3da5-44e1-9202-f99d5836efd0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:29.283860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:29.286818 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:29.293849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 452/911] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:29:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:29.296248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452386-de5e-428d-b397-1de66ce96432 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.296248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452386-de5e-428d-b397-1de66ce96432 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.296248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452386-de5e-428d-b397-1de66ce96432 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.296248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 453/912] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:29.296248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3452386-de5e-428d-b397-1de66ce96432 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:29.345345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6795f0-4197-4384-b712-6a50a14f2ecf None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:29.379311 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d7fd2-2da3-4b31-9403-207628451d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:29.380035 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d7fd2-2da3-4b31-9403-207628451d66 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.380325 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d7fd2-2da3-4b31-9403-207628451d66 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.380603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d7fd2-2da3-4b31-9403-207628451d66 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.381332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d7fd2-2da3-4b31-9403-207628451d66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:29.382703 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d7fd2-2da3-4b31-9403-207628451d66 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:29.388971 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 460/913] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:29:29 2022] GET /identity/v3/services => generated 1268 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:29:29.389965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c629af-0b29-4a78-99cd-6fad7a40dc7c None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.390257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c629af-0b29-4a78-99cd-6fad7a40dc7c None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.390537 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c629af-0b29-4a78-99cd-6fad7a40dc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.403978 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3452386-de5e-428d-b397-1de66ce96432 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:29.409560 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-a3c629af-0b29-4a78-99cd-6fad7a40dc7c None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:29.410871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-a3c629af-0b29-4a78-99cd-6fad7a40dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:29.452412 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:29.453543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.454067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.454588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.456841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:29.457494 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:29.459737 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:29.467290 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 454/914] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:29.477175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:29.492573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 461/915] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:29.521102 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c629af-0b29-4a78-99cd-6fad7a40dc7c None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:29.597122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5473f8-24a6-4c59-84f1-3641b52169b5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:29.632887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c629af-0b29-4a78-99cd-6fad7a40dc7c None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:29.638979 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:29.640100 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.640391 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.640934 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.641900 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:29.642251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:29.643678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:29.649327 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 455/916] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:29.655477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:29.671365 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b3b8b-8a71-432f-94fe-c6f97446a4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:29.672366 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3b8b-8a71-432f-94fe-c6f97446a4cc None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.672845 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3b8b-8a71-432f-94fe-c6f97446a4cc None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.673354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3b8b-8a71-432f-94fe-c6f97446a4cc None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.674854 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b3b8b-8a71-432f-94fe-c6f97446a4cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:29.676452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b3b8b-8a71-432f-94fe-c6f97446a4cc None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:29.694495 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 462/917] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:29:29 2022] POST /identity/v3/services => generated 241 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:29:29.775178 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffde90-4e00-4c04-b92e-f1a10308027e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:29.820834 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b44a9-2a83-406a-b985-6d6fb2e933f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:29.821542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b44a9-2a83-406a-b985-6d6fb2e933f0 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:29.821827 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b44a9-2a83-406a-b985-6d6fb2e933f0 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:29.822101 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b44a9-2a83-406a-b985-6d6fb2e933f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:29.822857 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b44a9-2a83-406a-b985-6d6fb2e933f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:29.824317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b44a9-2a83-406a-b985-6d6fb2e933f0 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:29.851997 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 456/918] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:29:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:31.243913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94487ad7-f63d-40a3-adca-4881580a9503 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:31.244951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94487ad7-f63d-40a3-adca-4881580a9503 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:31.245475 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94487ad7-f63d-40a3-adca-4881580a9503 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:31.247305 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 463/919] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:31.254858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5293bf1-5532-409c-b9d9-a66085f4e6b8 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:31.255361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5293bf1-5532-409c-b9d9-a66085f4e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:31.255453 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5293bf1-5532-409c-b9d9-a66085f4e6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:31.275056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-c5293bf1-5532-409c-b9d9-a66085f4e6b8 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:31.276259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-c5293bf1-5532-409c-b9d9-a66085f4e6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:31.349764 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 457/920] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:31.372032 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94487ad7-f63d-40a3-adca-4881580a9503 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:31.477504 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94487ad7-f63d-40a3-adca-4881580a9503 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:31.513478 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92aa09-3f33-4303-b347-d63edf05c4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:31.514726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92aa09-3f33-4303-b347-d63edf05c4f3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:31.515256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92aa09-3f33-4303-b347-d63edf05c4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:31.515759 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92aa09-3f33-4303-b347-d63edf05c4f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:31.529297 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92aa09-3f33-4303-b347-d63edf05c4f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-1e92aa09-3f33-4303-b347-d63edf05c4f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:31.538150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:31.539256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:31.540165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:31.543208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 464/921] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:29:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:31.549579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5293bf1-5532-409c-b9d9-a66085f4e6b8 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:31.651419 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5293bf1-5532-409c-b9d9-a66085f4e6b8 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:31.686103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d86a0b-b7f2-4da2-9e5e-19c9f2e71c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:31.686796 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d86a0b-b7f2-4da2-9e5e-19c9f2e71c95 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:31.687090 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d86a0b-b7f2-4da2-9e5e-19c9f2e71c95 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:31.687369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d86a0b-b7f2-4da2-9e5e-19c9f2e71c95 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:31.688138 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d86a0b-b7f2-4da2-9e5e-19c9f2e71c95 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:31.688454 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d86a0b-b7f2-4da2-9e5e-19c9f2e71c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:31.691843 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d86a0b-b7f2-4da2-9e5e-19c9f2e71c95 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:31.699958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 458/922] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:29:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:31.706322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92aa09-3f33-4303-b347-d63edf05c4f3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:31.720871 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37afb53-372f-4df9-97c2-9a4a81f7e927 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:31.721464 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37afb53-372f-4df9-97c2-9a4a81f7e927 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:31.721968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37afb53-372f-4df9-97c2-9a4a81f7e927 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:31.723761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 459/923] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:31.729203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67ddf-9fe5-46fa-91c9-6a122731f812 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:31.729687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67ddf-9fe5-46fa-91c9-6a122731f812 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:31.730187 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67ddf-9fe5-46fa-91c9-6a122731f812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:31.748037 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-d2e67ddf-9fe5-46fa-91c9-6a122731f812 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:31.749580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-d2e67ddf-9fe5-46fa-91c9-6a122731f812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:31.817406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92aa09-3f33-4303-b347-d63edf05c4f3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:31.829144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 460/924] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:31.847088 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e67ddf-9fe5-46fa-91c9-6a122731f812 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:31.852586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:31.853557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:31.853945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:31.854326 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:31.866280 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:31.867891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:31.885761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:31.886861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:31.888145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:31.888145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:31.888145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:31.888145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:31.888145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 465/925] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:31.893162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:31.955407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e67ddf-9fe5-46fa-91c9-6a122731f812 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:31.994472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4abb27-43aa-4a27-a6c3-02fc9159fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:31.995286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abb27-43aa-4a27-a6c3-02fc9159fb69 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:31.995575 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abb27-43aa-4a27-a6c3-02fc9159fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:31.995856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abb27-43aa-4a27-a6c3-02fc9159fb69 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:31.996523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4abb27-43aa-4a27-a6c3-02fc9159fb69 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:31.996634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef4b3-d802-4612-a46a-8693f1585305 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:31.997945 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4abb27-43aa-4a27-a6c3-02fc9159fb69 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5e4abb27-43aa-4a27-a6c3-02fc9159fb69 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:32.010399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:32.011492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.012381 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 461/926] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:32.013879 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427edcc7-e3e6-4819-8666-2234efa9f237 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.013879 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427edcc7-e3e6-4819-8666-2234efa9f237 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.013879 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427edcc7-e3e6-4819-8666-2234efa9f237 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.014921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 462/927] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:32.017967 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427edcc7-e3e6-4819-8666-2234efa9f237 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.039292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81fabc8-b93d-4087-acfa-22961b5255cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.040216 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fabc8-b93d-4087-acfa-22961b5255cd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.040674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fabc8-b93d-4087-acfa-22961b5255cd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.040952 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81fabc8-b93d-4087-acfa-22961b5255cd None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.041732 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fabc8-b93d-4087-acfa-22961b5255cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:32.042041 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fabc8-b93d-4087-acfa-22961b5255cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.043415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81fabc8-b93d-4087-acfa-22961b5255cd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.060029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 466/928] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:32.060029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a0037-11e5-4184-a236-5fdc6ce4d0a7 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.060029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a0037-11e5-4184-a236-5fdc6ce4d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.060029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a0037-11e5-4184-a236-5fdc6ce4d0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.073586 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-f93a0037-11e5-4184-a236-5fdc6ce4d0a7 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:32.074784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-f93a0037-11e5-4184-a236-5fdc6ce4d0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:32.127415 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427edcc7-e3e6-4819-8666-2234efa9f237 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.148961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 467/929] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:32.149640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a0037-11e5-4184-a236-5fdc6ce4d0a7 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.170801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a24ff-06b8-4f6f-9488-a1c94231b43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.171479 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a24ff-06b8-4f6f-9488-a1c94231b43d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.171763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a24ff-06b8-4f6f-9488-a1c94231b43d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.172044 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a24ff-06b8-4f6f-9488-a1c94231b43d None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.172759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a24ff-06b8-4f6f-9488-a1c94231b43d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:32.173123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a24ff-06b8-4f6f-9488-a1c94231b43d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.175107 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a24ff-06b8-4f6f-9488-a1c94231b43d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.179674 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 463/930] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:32.182785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6d0c2-f10d-4f3e-a2e8-c979059a9a15 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.182785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6d0c2-f10d-4f3e-a2e8-c979059a9a15 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.182785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6d0c2-f10d-4f3e-a2e8-c979059a9a15 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.182785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 464/931] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:32.183320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6d0c2-f10d-4f3e-a2e8-c979059a9a15 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.246527 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a0037-11e5-4184-a236-5fdc6ce4d0a7 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.277511 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6d0c2-f10d-4f3e-a2e8-c979059a9a15 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.279180 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.279993 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.280279 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.280560 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.287952 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.289431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:32.304528 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:32.305737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:32.306784 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 468/932] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:32.307448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.311118 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a45b12-ff50-4170-841f-dc5d24c5f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.311918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45b12-ff50-4170-841f-dc5d24c5f86f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.312199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45b12-ff50-4170-841f-dc5d24c5f86f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.312471 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45b12-ff50-4170-841f-dc5d24c5f86f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.313169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a45b12-ff50-4170-841f-dc5d24c5f86f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.314546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a45b12-ff50-4170-841f-dc5d24c5f86f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-c5a45b12-ff50-4170-841f-dc5d24c5f86f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:32.326193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:32.327434 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.328350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 465/933] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:32.329487 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667da114-6bb3-4a59-a6e4-1f7123ca0b17 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.329487 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667da114-6bb3-4a59-a6e4-1f7123ca0b17 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.329487 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667da114-6bb3-4a59-a6e4-1f7123ca0b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.344985 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-667da114-6bb3-4a59-a6e4-1f7123ca0b17 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:32.346728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-667da114-6bb3-4a59-a6e4-1f7123ca0b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:32.400587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f62c7d-96d2-4840-8471-e39fb775ddb9 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.409664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 466/934] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:32.410476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667da114-6bb3-4a59-a6e4-1f7123ca0b17 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.434369 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.435032 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.435321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.435597 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.436315 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:32.436620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.438019 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.442896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 469/935] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:32.443499 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.504070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667da114-6bb3-4a59-a6e4-1f7123ca0b17 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.536805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.537504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.537793 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.538073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.538919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:32.539232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.540441 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b2c0b-7333-417d-8b52-43c231ebefc4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.540664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.561589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 467/936] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:32.561589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.572577 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.573274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.573584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.573861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.574574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:32.574881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.576242 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.581253 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 470/937] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:32.581998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.650869 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374350ae-ce1a-4799-828f-bed3d2b55339 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.674602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3f172-a52c-4d8d-9b8b-bdd6c9086ff4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.682860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.683652 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.683934 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.684210 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.684867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.686257 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:32.697665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:32.698919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.700062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 468/938] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:32.701147 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.713456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4c061-4766-465c-863c-76ec3b90cd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.714108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4c061-4766-465c-863c-76ec3b90cd63 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.714399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4c061-4766-465c-863c-76ec3b90cd63 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.714681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4c061-4766-465c-863c-76ec3b90cd63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.715464 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4c061-4766-465c-863c-76ec3b90cd63 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.716838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4c061-4766-465c-863c-76ec3b90cd63 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.734406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 471/939] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:29:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:32.735192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4c061-4766-465c-863c-76ec3b90cd63 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.807312 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89a16e-78e6-4c11-95cb-9e77d0451ea3 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.848760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.849667 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.849961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.850247 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.853298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4c061-4766-465c-863c-76ec3b90cd63 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:32.863435 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.864927 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:32.883029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:32.884302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:32.885348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:32.885348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:32.885348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:32.885348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:32.885348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 469/940] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:32.885678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:32.896000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:32.896843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:32.897279 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:32.897663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:32.898514 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:32.898933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:32.900453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:32.906169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 472/941] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:32.907135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:33.000923 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76ff2d-d523-444e-a164-30b7515831e4 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:33.013910 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e465e2-d8fe-4084-b555-cf57b4123b20 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:33.037407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:33.038484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:33.038989 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:33.039472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:33.040382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] RBAC: Adding query filter params (name=compute) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:33.040893 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:33.042538 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:33.046737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:33.052546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 470/942] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:33.052703 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:33.052703 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:33.052703 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:33.052980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:33.053124 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:33.053124 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:33.053124 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:33.069130 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 473/943] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:33.070039 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:33.154789 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab754f80-8664-43e1-945f-e8b070538051 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:33.168647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6f9cb-05d2-42d7-b10a-fe5705af26a5 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:33.187424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:33.188080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:33.188362 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:33.188639 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:33.189459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] RBAC: Adding query filter params (interface=public, service_id=a94f77bd804e4f9ca1f82dfba4434af7, region_id=RegionOne) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:33.189800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:33.191170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:33.200563 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 471/944] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:29:33 2022] GET /identity/v3/endpoints?service_id=a94f77bd804e4f9ca1f82dfba4434af7&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:33.200563 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:33.204129 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:33.204801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:33.205626 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:33.206015 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:33.206737 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:33.207043 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:33.208407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:33.212841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 474/945] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:33.213652 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:33.323745 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e8793-5aaa-4990-ac3f-260a08139b5b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:33.326673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11f612-78c9-4fb7-a6f7-52fd94fa56b6 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:33.361125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:33.361981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:33.362266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:33.362544 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:33.364787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69a9dd-d33a-460a-8894-d71a0f746e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:33.365552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69a9dd-d33a-460a-8894-d71a0f746e56 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:33.365838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69a9dd-d33a-460a-8894-d71a0f746e56 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:33.366112 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69a9dd-d33a-460a-8894-d71a0f746e56 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:33.366816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69a9dd-d33a-460a-8894-d71a0f746e56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:33.368241 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69a9dd-d33a-460a-8894-d71a0f746e56 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:33.370290 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:33.371786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:33.376576 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 475/946] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:29:33 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:33.387585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:33.388860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:33.390285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 472/947] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:33.397822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69a9dd-d33a-460a-8894-d71a0f746e56 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:33.512604 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69a9dd-d33a-460a-8894-d71a0f746e56 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:33.551655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd05e61-00c8-469f-9fd0-bb89b37d6adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:33.552719 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05e61-00c8-469f-9fd0-bb89b37d6adf None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:33.553246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05e61-00c8-469f-9fd0-bb89b37d6adf None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:33.553732 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd05e61-00c8-469f-9fd0-bb89b37d6adf None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:33.554767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd05e61-00c8-469f-9fd0-bb89b37d6adf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:33.555295 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd05e61-00c8-469f-9fd0-bb89b37d6adf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:33.556991 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd05e61-00c8-469f-9fd0-bb89b37d6adf None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:33.574533 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 476/948] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:33.580940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:33.686210 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00197816-77de-465a-9d18-472f395e93b7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:33.722034 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cee27-d541-46fb-a046-f78fa40f54c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:33.722938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cee27-d541-46fb-a046-f78fa40f54c8 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:33.723229 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cee27-d541-46fb-a046-f78fa40f54c8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:33.723513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cee27-d541-46fb-a046-f78fa40f54c8 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/8db5070e21174e5593f244bdb2c52224/roles/87f96af882934745afa3b46c6d539918` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:33.725814 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cee27-d541-46fb-a046-f78fa40f54c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=8db5070e21174e5593f244bdb2c52224, role_id=87f96af882934745afa3b46c6d539918)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:33.727267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cee27-d541-46fb-a046-f78fa40f54c8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:33.749238 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 473/949] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:29:33 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/8db5070e21174e5593f244bdb2c52224/roles/87f96af882934745afa3b46c6d539918 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:29:35.048689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace657a8-a3cd-4257-be22-003f0b119dcc None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.049795 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace657a8-a3cd-4257-be22-003f0b119dcc None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.049879 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace657a8-a3cd-4257-be22-003f0b119dcc None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.051487 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 477/950] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:35.057135 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e885cbb-098b-4b59-94b7-376fecbf7317 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.057541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e885cbb-098b-4b59-94b7-376fecbf7317 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.057752 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e885cbb-098b-4b59-94b7-376fecbf7317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.075309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-2e885cbb-098b-4b59-94b7-376fecbf7317 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:35.076512 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-2e885cbb-098b-4b59-94b7-376fecbf7317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:35.146530 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 474/951] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:35.169874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace657a8-a3cd-4257-be22-003f0b119dcc None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:35.275175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace657a8-a3cd-4257-be22-003f0b119dcc None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:35.314042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:35.314668 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.314956 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.315240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.315975 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:35.317461 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:35.318871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: WARNING keystone.server.flask.common [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 15:29:35.343913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.sql.core [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:29:35.343913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 15:29:35.343913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [parameters: {'id': '90a29d3024f74afcb35f3b4201424a25', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 15:29:35.343913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=206280) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 90a29d3024f74afcb35f3b4201424a25.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 90a29d3024f74afcb35f3b4201424a25. May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application conn.query(q) May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:29:35.361099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result.read() May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 09 15:29:35.363332 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return project_ref.to_dict() May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application next(self.gen) May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application yield resource May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application next(self.gen) May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise self.value May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application session.commit() May 09 15:29:35.365823 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.session.flush() May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application compat.raise_( May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:29:35.368099 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:29:35.370507 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application conn.query(q) May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result.read() May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:29:35.378601 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [parameters: {'id': '90a29d3024f74afcb35f3b4201424a25', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:35.380871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:35.382960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.Conflict( May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 90a29d3024f74afcb35f3b4201424a25. May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:35.385246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 478/952] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 9 15:29:35 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:35.386624 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e885cbb-098b-4b59-94b7-376fecbf7317 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:35.473980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e885cbb-098b-4b59-94b7-376fecbf7317 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:35.508460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ee47d-9766-4782-8c8d-5a151639fe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:35.509291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee47d-9766-4782-8c8d-5a151639fe1b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.509583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee47d-9766-4782-8c8d-5a151639fe1b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.509869 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ee47d-9766-4782-8c8d-5a151639fe1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.516805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ee47d-9766-4782-8c8d-5a151639fe1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:35.518842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ee47d-9766-4782-8c8d-5a151639fe1b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-062ee47d-9766-4782-8c8d-5a151639fe1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:35.532953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:35.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:35.538102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 475/953] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:35.542216 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:35.562452 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5759ab-02dc-4b48-b552-1db618edccc5 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.562787 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5759ab-02dc-4b48-b552-1db618edccc5 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.563013 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5759ab-02dc-4b48-b552-1db618edccc5 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.565079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 476/954] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:35.571064 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640a58e-24a1-4978-9096-65e576c9cf68 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.571779 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640a58e-24a1-4978-9096-65e576c9cf68 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.572073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640a58e-24a1-4978-9096-65e576c9cf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.594179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-4640a58e-24a1-4978-9096-65e576c9cf68 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:35.596478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-4640a58e-24a1-4978-9096-65e576c9cf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:35.666730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68cf67-91ff-4a1d-a932-fced1aae96d9 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:35.693154 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 477/955] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:35.710041 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b93c7-863d-45e6-aee6-a042e1e81127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:35.710718 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b93c7-863d-45e6-aee6-a042e1e81127 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.711005 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b93c7-863d-45e6-aee6-a042e1e81127 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.711290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b93c7-863d-45e6-aee6-a042e1e81127 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.712071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b93c7-863d-45e6-aee6-a042e1e81127 None admin] RBAC: Adding query filter params (name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:35.712385 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b93c7-863d-45e6-aee6-a042e1e81127 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:35.714090 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b93c7-863d-45e6-aee6-a042e1e81127 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:35.716017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640a58e-24a1-4978-9096-65e576c9cf68 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:35.739965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 479/956] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:35 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:35.759890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05dcf-4731-4ec4-b288-5c07a5a5a59c None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.759890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05dcf-4731-4ec4-b288-5c07a5a5a59c None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.759890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05dcf-4731-4ec4-b288-5c07a5a5a59c None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.759890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 480/957] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:35.768761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05c74a-aca1-419d-a1e3-1566cb16c5e7 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.768761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05c74a-aca1-419d-a1e3-1566cb16c5e7 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.768761 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05c74a-aca1-419d-a1e3-1566cb16c5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.786472 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-7a05c74a-aca1-419d-a1e3-1566cb16c5e7 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:35.788454 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-7a05c74a-aca1-419d-a1e3-1566cb16c5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:35.838427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640a58e-24a1-4978-9096-65e576c9cf68 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:35.864153 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 481/958] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:35.884703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66d3e4-3902-4c56-a146-d7e755b3687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:35.885449 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66d3e4-3902-4c56-a146-d7e755b3687f None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:35.885735 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66d3e4-3902-4c56-a146-d7e755b3687f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:35.886010 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66d3e4-3902-4c56-a146-d7e755b3687f None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:35.886763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66d3e4-3902-4c56-a146-d7e755b3687f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:35.888215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66d3e4-3902-4c56-a146-d7e755b3687f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:35.889886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05c74a-aca1-419d-a1e3-1566cb16c5e7 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:35.913714 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 478/959] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:29:35 2022] POST /identity/v3/services => generated 242 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:29:36.010867 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05c74a-aca1-419d-a1e3-1566cb16c5e7 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:36.058484 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:36.059318 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.059608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.059893 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.060579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:36.062035 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:36.069385 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b979-a06c-4f22-8f52-00e1623cef7d None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.069941 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b979-a06c-4f22-8f52-00e1623cef7d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.069941 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b979-a06c-4f22-8f52-00e1623cef7d None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.073172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 479/960] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:36.075417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:36.076489 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:36.077466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 482/961] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:36.078348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97696ecc-bd3a-4c6c-8683-e9a532553228 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.078348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97696ecc-bd3a-4c6c-8683-e9a532553228 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.078494 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97696ecc-bd3a-4c6c-8683-e9a532553228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.083174 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:36.110236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-97696ecc-bd3a-4c6c-8683-e9a532553228 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:36.112371 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-97696ecc-bd3a-4c6c-8683-e9a532553228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:36.187166 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c8227-1819-4a9e-8f3e-1d549175400b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:36.191175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 480/962] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:36.215130 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97696ecc-bd3a-4c6c-8683-e9a532553228 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:36.223286 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:36.223965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.224259 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.224532 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.225308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:36.225623 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:36.227004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:36.232569 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 483/963] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:36.238585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:36.325314 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97696ecc-bd3a-4c6c-8683-e9a532553228 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:36.342197 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452112ad-120a-4029-ba7d-6dee99a8b708 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:36.359644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67d091-2877-4778-9045-5df8bb885e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:36.360387 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67d091-2877-4778-9045-5df8bb885e23 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.360670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67d091-2877-4778-9045-5df8bb885e23 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.360948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67d091-2877-4778-9045-5df8bb885e23 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.372138 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67d091-2877-4778-9045-5df8bb885e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:36.378212 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:36.378887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.379175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.379456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.380187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:36.380501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-2f67d091-2877-4778-9045-5df8bb885e23 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:36.381483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:36.382486 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:36.383409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:36.384115 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:36.384199 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 481/964] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:29:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:36.388685 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67d091-2877-4778-9045-5df8bb885e23 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:36.391621 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 484/965] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:36.397067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:36.503760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67d091-2877-4778-9045-5df8bb885e23 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:36.510368 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad9540-b2d6-433a-ad5f-1ff547891625 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:36.536843 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:36.537611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.537900 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.538177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.539036 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:36.539353 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:36.540722 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:36.549715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 482/966] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:29:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:36.551255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff750c-fe02-4f85-9e33-1c6dac5928a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:36.551921 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff750c-fe02-4f85-9e33-1c6dac5928a8 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.552211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff750c-fe02-4f85-9e33-1c6dac5928a8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.552493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff750c-fe02-4f85-9e33-1c6dac5928a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.553313 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff750c-fe02-4f85-9e33-1c6dac5928a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:36.554700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff750c-fe02-4f85-9e33-1c6dac5928a8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:36.554922 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:36.585143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 485/967] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:29:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:29:36.672255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f045e-8123-4b2c-946c-01a230412b53 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:36.715446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:36.716278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.716564 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.716840 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.729355 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:36.730838 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:36.750243 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:36.752523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:36.754886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:36.754886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:36.754886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:36.754886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:36.755427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 483/968] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:36.758435 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff750c-fe02-4f85-9e33-1c6dac5928a8 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:36.864509 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff750c-fe02-4f85-9e33-1c6dac5928a8 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:36.902152 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:36.902833 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:36.903115 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:36.903392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:36.904128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:36.904437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:36.905847 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:36.922992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 486/969] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:36.924867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:37.032480 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62083ca-08d5-45ea-8264-dee861573fc7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:37.067509 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:37.068328 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:37.068609 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:37.068884 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:37.077102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:37.078569 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:37.099481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:37.102440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:37.110196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 484/970] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:29:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:37.111741 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:37.207735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbe4b-a53c-42ad-805f-b314e24472f1 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:37.242838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:37.243516 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:37.243799 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:37.244080 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:37.244937 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:37.245279 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:37.246781 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:37.268944 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 487/971] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:29:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:37.275787 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:37.404271 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b1fe3-9fde-4da5-a27c-c205d05a7ca8 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:37.439343 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92374a3-aa38-43a8-94d5-73c6e8b90192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:37.440529 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92374a3-aa38-43a8-94d5-73c6e8b90192 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:37.440813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92374a3-aa38-43a8-94d5-73c6e8b90192 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:37.441996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92374a3-aa38-43a8-94d5-73c6e8b90192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:37.515467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92374a3-aa38-43a8-94d5-73c6e8b90192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:37.516871 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92374a3-aa38-43a8-94d5-73c6e8b90192 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:37.639242 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 485/972] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:37 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:37.640740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:37.752707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81f5a9-afbb-40f4-bc89-f3c0d6343e2c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:37.787650 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:37.788436 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:37.788726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:37.789000 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:37.800417 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:37.805065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:37.820923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:37.824309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:37.826758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:37.826758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:37.826758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:37.826758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:37.827313 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 488/973] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:37.833072 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92374a3-aa38-43a8-94d5-73c6e8b90192 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:37.938839 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92374a3-aa38-43a8-94d5-73c6e8b90192 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:37.980456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072d37f-15f5-4c56-8ca5-2180c230234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:37.981174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072d37f-15f5-4c56-8ca5-2180c230234d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:37.981452 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072d37f-15f5-4c56-8ca5-2180c230234d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:37.985226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072d37f-15f5-4c56-8ca5-2180c230234d None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:37.985226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072d37f-15f5-4c56-8ca5-2180c230234d None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:37.985226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072d37f-15f5-4c56-8ca5-2180c230234d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:37.985226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072d37f-15f5-4c56-8ca5-2180c230234d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:38.005286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 486/974] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:38.006939 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:38.111803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c9c4-3906-4a88-8938-a36ff32011fb None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:38.147582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c29ad0-a6c1-4198-a6f5-0fc52076116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:38.148388 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c29ad0-a6c1-4198-a6f5-0fc52076116a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.148671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c29ad0-a6c1-4198-a6f5-0fc52076116a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.148945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c29ad0-a6c1-4198-a6f5-0fc52076116a None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.155810 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c29ad0-a6c1-4198-a6f5-0fc52076116a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:38.157442 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c29ad0-a6c1-4198-a6f5-0fc52076116a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-89c29ad0-a6c1-4198-a6f5-0fc52076116a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:38.171587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:38.174010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:38.176344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:38.176344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:38.176344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:38.176344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:38.176344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:38.176344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:38.176344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:38.176344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:38.177467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735198b-3c36-45fe-bb73-23c3a7706eab None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.177467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735198b-3c36-45fe-bb73-23c3a7706eab None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.177467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735198b-3c36-45fe-bb73-23c3a7706eab None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.177467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 487/976] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:38.178350 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 489/976] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:38.185062 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735198b-3c36-45fe-bb73-23c3a7706eab None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:38.185319 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250c82-997a-48d6-83eb-ecd7dd55548c None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.185319 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250c82-997a-48d6-83eb-ecd7dd55548c None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.185319 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03250c82-997a-48d6-83eb-ecd7dd55548c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.203511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-03250c82-997a-48d6-83eb-ecd7dd55548c None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:38.204864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-03250c82-997a-48d6-83eb-ecd7dd55548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:38.286299 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735198b-3c36-45fe-bb73-23c3a7706eab None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:38.293638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 490/977] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:38.320414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03250c82-997a-48d6-83eb-ecd7dd55548c None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:38.326898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:38.327572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.327856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.328135 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.328940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:38.329283 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:38.331287 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:38.348521 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 488/978] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:38.349840 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:38.427072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03250c82-997a-48d6-83eb-ecd7dd55548c None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:38.450445 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab68c61-9632-4fce-8a87-1295ea07f060 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:38.462909 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a619294-99f8-4149-9b83-35525d84abff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:38.463595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a619294-99f8-4149-9b83-35525d84abff None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.463881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a619294-99f8-4149-9b83-35525d84abff None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.464163 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a619294-99f8-4149-9b83-35525d84abff None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.465068 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a619294-99f8-4149-9b83-35525d84abff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:38.466617 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a619294-99f8-4149-9b83-35525d84abff None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:38.479843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.sql.core [None req-8a619294-99f8-4149-9b83-35525d84abff None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:29:38.479843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 15:29:38.479843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [parameters: {'user_id': '2a995ae467ef4837a318a0127b75ecb8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 15:29:38.479843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=206280) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:29:38.485475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107513d9-4181-4ad7-84a8-d02ee9bfa019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:38.486172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107513d9-4181-4ad7-84a8-d02ee9bfa019 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.487002 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107513d9-4181-4ad7-84a8-d02ee9bfa019 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.487283 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107513d9-4181-4ad7-84a8-d02ee9bfa019 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-8a619294-99f8-4149-9b83-35525d84abff None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application conn.query(q) May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result.read() May 09 15:29:38.488001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:29:38.490184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application next(self.gen) May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application yield resource May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application next(self.gen) May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise self.value May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application session.commit() May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:29:38.493402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.session.flush() May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application compat.raise_( May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:29:38.496057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result = connection._execute_20( May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:29:38.498453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application conn.query(q) May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application result.read() May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:29:38.500723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [parameters: {'user_id': '2a995ae467ef4837a318a0127b75ecb8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:38.506177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 09 15:29:38.507085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 09 15:29:38.508091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 09 15:29:38.508091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 09 15:29:38.508091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:29:38.508091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:29:38.508091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 15:29:38.508091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:38.508446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107513d9-4181-4ad7-84a8-d02ee9bfa019 None admin] RBAC: Adding query filter params (role.id=87f96af882934745afa3b46c6d539918, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=8db5070e21174e5593f244bdb2c52224) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:38.508446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107513d9-4181-4ad7-84a8-d02ee9bfa019 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:38.508446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107513d9-4181-4ad7-84a8-d02ee9bfa019 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:38.508681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 491/979] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:29:38 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:38.508681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860f6c-6b65-4c74-aaa6-d4c3ded3b8d1 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.508681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860f6c-6b65-4c74-aaa6-d4c3ded3b8d1 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.508681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7860f6c-6b65-4c74-aaa6-d4c3ded3b8d1 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.508681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 492/980] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:38.509013 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 489/981] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:38 2022] GET /identity/v3/role_assignments?user.id=8db5070e21174e5593f244bdb2c52224&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=87f96af882934745afa3b46c6d539918 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:38.509122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860f6c-6b65-4c74-aaa6-d4c3ded3b8d1 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:38.509220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bede4f-ecac-4f66-adf8-ca2c0f26807f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.509220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bede4f-ecac-4f66-adf8-ca2c0f26807f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.509220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bede4f-ecac-4f66-adf8-ca2c0f26807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.534924 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-c5bede4f-ecac-4f66-adf8-ca2c0f26807f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:38.536572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-c5bede4f-ecac-4f66-adf8-ca2c0f26807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:38.627148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 490/982] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:38.643425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7860f6c-6b65-4c74-aaa6-d4c3ded3b8d1 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:38.656338 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bede4f-ecac-4f66-adf8-ca2c0f26807f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:38.689199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:38.690430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.691103 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.692404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.704255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:38.706797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:38.728040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:38.729204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:38.730250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 493/983] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:38.737652 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:38.786025 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bede4f-ecac-4f66-adf8-ca2c0f26807f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:38.831128 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:38.832315 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.832602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.832880 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] PATH_INFO: `/v3/services/compute` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.833610 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:38.835661 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:38.847483 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1ec56-7639-4542-8b8b-b4350c544ce0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:38.852126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:38.853319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:38.854255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 491/984] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:38.860043 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:38.889876 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0e1e8-62d9-4861-ae5d-b1fe7efde62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:38.890720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0e1e8-62d9-4861-ae5d-b1fe7efde62a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:38.891314 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0e1e8-62d9-4861-ae5d-b1fe7efde62a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:38.891607 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0e1e8-62d9-4861-ae5d-b1fe7efde62a None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:38.893327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0e1e8-62d9-4861-ae5d-b1fe7efde62a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:38.893643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0e1e8-62d9-4861-ae5d-b1fe7efde62a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:38.896134 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0e1e8-62d9-4861-ae5d-b1fe7efde62a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:38.943986 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 494/985] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:29:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:38.967519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4b80e-932e-4bf5-9b63-d9baac3a877a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.006213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.006969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.007256 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.007539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.008352 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] RBAC: Adding query filter params (name=compute) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:39.008664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.010182 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.016588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 492/986] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:39.019740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d0c2-cb59-43b0-9ebc-68d952880b0c None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.020032 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d0c2-cb59-43b0-9ebc-68d952880b0c None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.020314 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d0c2-cb59-43b0-9ebc-68d952880b0c None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.022170 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 495/987] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:39.023252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:39.028070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16570e6-8161-4c8a-b8a2-7bf9283e90dd None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.028361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16570e6-8161-4c8a-b8a2-7bf9283e90dd None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.028656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16570e6-8161-4c8a-b8a2-7bf9283e90dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.046246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-a16570e6-8161-4c8a-b8a2-7bf9283e90dd None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:39.047815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-a16570e6-8161-4c8a-b8a2-7bf9283e90dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:39.135770 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 496/988] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:39.151225 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4321844-517d-4ae0-90f1-5bb86324528a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.156571 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16570e6-8161-4c8a-b8a2-7bf9283e90dd None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:39.195602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.196260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.196501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.196779 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.197543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] RBAC: Adding query filter params (type=compute) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:39.197859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.199241 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.205869 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 493/989] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:39.212029 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:39.273937 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16570e6-8161-4c8a-b8a2-7bf9283e90dd None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.313481 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.314292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.315200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.315511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.316192 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.317602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.326036 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808dcfa-e5a4-44d4-ad63-4adc1a3cc15d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:39.329634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:39.330708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:39.337086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 497/990] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:39.338190 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:39.365791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.366457 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.366746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.367028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.367849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] RBAC: Adding query filter params (interface=public, service_id=a26a3088c328450d882a069959121c42, region_id=RegionOne) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:39.368153 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.369564 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.374570 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 494/991] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:29:39 2022] GET /identity/v3/endpoints?service_id=a26a3088c328450d882a069959121c42&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:39.380522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:39.446517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41130fb-4a39-457a-bfb0-c2dbcb784688 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.478913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.479117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237bc06-a0e9-4eaa-ae5a-4f6663b551ef None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.479581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.479866 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.480146 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.480861 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:39.481208 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.482842 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.488271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 498/992] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:39.493852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:39.513247 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59fe4a-90ac-4fa3-b957-df61f127021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.513918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59fe4a-90ac-4fa3-b957-df61f127021b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.514213 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59fe4a-90ac-4fa3-b957-df61f127021b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.514508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59fe4a-90ac-4fa3-b957-df61f127021b None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.515191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59fe4a-90ac-4fa3-b957-df61f127021b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.516554 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59fe4a-90ac-4fa3-b957-df61f127021b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.522585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 495/993] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:29:39 2022] GET /identity/v3/services => generated 1727 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:29:39.604354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a55eb-ac6c-4909-b470-51ba26e1f7a8 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.640148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.641216 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.641721 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.642152 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.643177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:39.643656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.645245 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.656148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 499/994] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:39.657828 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59fe4a-90ac-4fa3-b957-df61f127021b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:39.763176 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59fe4a-90ac-4fa3-b957-df61f127021b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.795697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcba4b-2145-4c69-9396-c2c1d1b9d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.796364 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcba4b-2145-4c69-9396-c2c1d1b9d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.796647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcba4b-2145-4c69-9396-c2c1d1b9d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.796923 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcba4b-2145-4c69-9396-c2c1d1b9d2e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.797742 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcba4b-2145-4c69-9396-c2c1d1b9d2e2 None admin] RBAC: Adding query filter params (interface=admin, service_id=a94f77bd804e4f9ca1f82dfba4434af7, region_id=RegionOne) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:39.798059 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcba4b-2145-4c69-9396-c2c1d1b9d2e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.800038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcba4b-2145-4c69-9396-c2c1d1b9d2e2 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.805267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 496/995] 10.222.0.13 () {56 vars in 1249 bytes} [Mon May 9 15:29:39 2022] GET /identity/v3/endpoints?service_id=a94f77bd804e4f9ca1f82dfba4434af7&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:39.810742 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:39.906573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11b7b7-f34b-4af0-8cd0-99c07736dadb None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:39.942120 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd26e4-7e23-4483-bc09-a431e2ee2318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:39.942789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd26e4-7e23-4483-bc09-a431e2ee2318 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:39.943072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd26e4-7e23-4483-bc09-a431e2ee2318 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:39.943342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd26e4-7e23-4483-bc09-a431e2ee2318 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:39.944037 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd26e4-7e23-4483-bc09-a431e2ee2318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:39.945437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd26e4-7e23-4483-bc09-a431e2ee2318 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:39.953078 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 500/996] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:29:39 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:29:40.854116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312f835-33d0-42f6-8926-1c3b7c907fc7 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:40.854689 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312f835-33d0-42f6-8926-1c3b7c907fc7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:40.854689 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312f835-33d0-42f6-8926-1c3b7c907fc7 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:40.856209 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 497/997] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:40.862205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9e682-33d2-451c-8606-2ab7580776a9 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:40.862863 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9e682-33d2-451c-8606-2ab7580776a9 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:40.863219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9e682-33d2-451c-8606-2ab7580776a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:40.880331 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-5af9e682-33d2-451c-8606-2ab7580776a9 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:40.881560 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-5af9e682-33d2-451c-8606-2ab7580776a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:40.945943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 501/998] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:40.967413 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312f835-33d0-42f6-8926-1c3b7c907fc7 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:41.069466 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312f835-33d0-42f6-8926-1c3b7c907fc7 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:41.115100 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:41.115843 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:41.116134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:41.116417 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] PATH_INFO: `/v3/services/image` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:41.117172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:41.118604 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:41.131295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:41.132465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:41.138456 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 498/999] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:29:40 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:41.139341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9e682-33d2-451c-8606-2ab7580776a9 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:41.235334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9e682-33d2-451c-8606-2ab7580776a9 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:41.269257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d2b6e-2b9e-43ea-a0e6-5065477e1180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:41.269936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d2b6e-2b9e-43ea-a0e6-5065477e1180 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:41.270322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d2b6e-2b9e-43ea-a0e6-5065477e1180 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:41.270608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d2b6e-2b9e-43ea-a0e6-5065477e1180 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:41.271328 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d2b6e-2b9e-43ea-a0e6-5065477e1180 None admin] RBAC: Adding query filter params (name=image) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:41.271641 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d2b6e-2b9e-43ea-a0e6-5065477e1180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:41.273031 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d2b6e-2b9e-43ea-a0e6-5065477e1180 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:41.282211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 502/1000] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:41.283911 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:41.381492 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05da47b-3029-41dc-a1ef-221f9066da51 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:41.413638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662dbe8-da43-4794-9e98-39c03ed14491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:41.414304 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dbe8-da43-4794-9e98-39c03ed14491 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:41.414591 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dbe8-da43-4794-9e98-39c03ed14491 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:41.414873 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662dbe8-da43-4794-9e98-39c03ed14491 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:41.415590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662dbe8-da43-4794-9e98-39c03ed14491 None admin] RBAC: Adding query filter params (type=image) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:41.415906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662dbe8-da43-4794-9e98-39c03ed14491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:41.417302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662dbe8-da43-4794-9e98-39c03ed14491 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:41.429072 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 499/1001] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:41.796899 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155eb35-7dbb-49be-bb1c-7d9124d01c7e None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:41.797258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155eb35-7dbb-49be-bb1c-7d9124d01c7e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:41.797511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155eb35-7dbb-49be-bb1c-7d9124d01c7e None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:41.798972 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 503/1002] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:41.804218 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991a3a-535a-4efa-a9e4-8a3b16ee1441 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:41.804513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991a3a-535a-4efa-a9e4-8a3b16ee1441 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:41.804799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991a3a-535a-4efa-a9e4-8a3b16ee1441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:41.823759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-2d991a3a-535a-4efa-a9e4-8a3b16ee1441 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:41.824947 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-2d991a3a-535a-4efa-a9e4-8a3b16ee1441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:41.890005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 500/1003] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:41.912803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155eb35-7dbb-49be-bb1c-7d9124d01c7e None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:42.013858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155eb35-7dbb-49be-bb1c-7d9124d01c7e None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:42.047941 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:42.048699 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.048985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.049292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] PATH_INFO: `/v3/services/compute` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.049974 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:42.051395 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:42.063936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:42.065731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:29:42.066645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:42.068232 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 504/1004] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:42.072154 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d991a3a-535a-4efa-a9e4-8a3b16ee1441 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:42.176446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d991a3a-535a-4efa-a9e4-8a3b16ee1441 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:42.210015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89805176-319f-476d-a671-f6767ec8c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:42.210690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89805176-319f-476d-a671-f6767ec8c779 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.210979 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89805176-319f-476d-a671-f6767ec8c779 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.211259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89805176-319f-476d-a671-f6767ec8c779 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.211976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89805176-319f-476d-a671-f6767ec8c779 None admin] RBAC: Adding query filter params (name=compute) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:42.212286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89805176-319f-476d-a671-f6767ec8c779 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:42.213689 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89805176-319f-476d-a671-f6767ec8c779 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:42.224555 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 501/1005] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:42.224745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:42.251103 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf726ef0-52ed-45b4-8978-5f5310e84fd9 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.251399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf726ef0-52ed-45b4-8978-5f5310e84fd9 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.251681 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf726ef0-52ed-45b4-8978-5f5310e84fd9 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.253801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 502/1006] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:42.258724 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317adcde-5a2a-4f60-90b2-8b22774c80a1 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.259014 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317adcde-5a2a-4f60-90b2-8b22774c80a1 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.259294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317adcde-5a2a-4f60-90b2-8b22774c80a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.275534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-317adcde-5a2a-4f60-90b2-8b22774c80a1 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:42.276727 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-317adcde-5a2a-4f60-90b2-8b22774c80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:42.328504 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4eaee-d8df-4c84-8f44-2bacb23a2f64 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:42.349102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 503/1007] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:42.361980 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:42.362646 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.362930 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.363205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.363921 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] RBAC: Adding query filter params (type=compute) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:42.364231 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:42.365636 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:42.374890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 505/1008] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:29:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:42.375056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317adcde-5a2a-4f60-90b2-8b22774c80a1 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:42.375611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:42.472039 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006504a-d452-4eeb-ac13-92d4e14d2408 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:42.475327 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317adcde-5a2a-4f60-90b2-8b22774c80a1 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:42.505689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7bdb9-f65d-469a-bd99-3835f1297579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:42.506396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7bdb9-f65d-469a-bd99-3835f1297579 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.506688 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7bdb9-f65d-469a-bd99-3835f1297579 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.506969 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7bdb9-f65d-469a-bd99-3835f1297579 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.507720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7bdb9-f65d-469a-bd99-3835f1297579 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:42.509117 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7bdb9-f65d-469a-bd99-3835f1297579 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:42.511802 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:42.512821 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.513316 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.513785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.514632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:42.516270 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:42.537068 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 506/1009] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:29:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:42.537291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:42.538585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:42.539699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 504/1010] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:29:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:42.540752 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7bdb9-f65d-469a-bd99-3835f1297579 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:42.657961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7bdb9-f65d-469a-bd99-3835f1297579 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:42.693795 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:42.694477 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.694799 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.695035 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.695766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:42.696075 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:42.697570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:42.708127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 507/1011] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:42.709762 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:42.815886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c540d-c26e-4b38-aac4-576289c967bc None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:42.850413 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5197f-fe99-4a58-af43-25e68cfcd238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:42.851086 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5197f-fe99-4a58-af43-25e68cfcd238 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.851374 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5197f-fe99-4a58-af43-25e68cfcd238 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.851651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5197f-fe99-4a58-af43-25e68cfcd238 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.852362 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5197f-fe99-4a58-af43-25e68cfcd238 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:42.852672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5197f-fe99-4a58-af43-25e68cfcd238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:42.854716 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5197f-fe99-4a58-af43-25e68cfcd238 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:42.860071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 505/1012] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:29:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:42.866204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:42.961657 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59a977-07ee-4f68-89ff-6d13a1c8702b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:42.994206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650005d1-3f30-4a77-a721-f8221eab2a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:42.994874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650005d1-3f30-4a77-a721-f8221eab2a28 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:42.995156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650005d1-3f30-4a77-a721-f8221eab2a28 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:42.995433 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650005d1-3f30-4a77-a721-f8221eab2a28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:42.996195 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650005d1-3f30-4a77-a721-f8221eab2a28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:42.997597 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650005d1-3f30-4a77-a721-f8221eab2a28 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:43.021071 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 508/1013] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:29:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:43.670435 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998ce8-7a74-48d6-9065-56f8de22f61d None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:43.670962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998ce8-7a74-48d6-9065-56f8de22f61d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:43.671045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84998ce8-7a74-48d6-9065-56f8de22f61d None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:43.672490 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 506/1014] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:43.677695 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59008fb4-c267-4c88-b89a-f5001fa13d63 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:43.677989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59008fb4-c267-4c88-b89a-f5001fa13d63 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:43.678269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59008fb4-c267-4c88-b89a-f5001fa13d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:43.695184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-59008fb4-c267-4c88-b89a-f5001fa13d63 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:43.696354 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-59008fb4-c267-4c88-b89a-f5001fa13d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:43.767523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 509/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:43 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:43.790028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998ce8-7a74-48d6-9065-56f8de22f61d None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:43.890827 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84998ce8-7a74-48d6-9065-56f8de22f61d None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:43.923461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83f4af-8bc9-4ff7-b6af-6b884c195594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:43.924584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83f4af-8bc9-4ff7-b6af-6b884c195594 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:43.924867 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83f4af-8bc9-4ff7-b6af-6b884c195594 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:43.925164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83f4af-8bc9-4ff7-b6af-6b884c195594 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:43.925970 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83f4af-8bc9-4ff7-b6af-6b884c195594 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:43.927252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83f4af-8bc9-4ff7-b6af-6b884c195594 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:43.949085 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 507/1016] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:29:43 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:44.879334 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61692-b36d-4bc8-bf2d-f3fd4ea46da1 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:44.879864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61692-b36d-4bc8-bf2d-f3fd4ea46da1 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:44.879864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61692-b36d-4bc8-bf2d-f3fd4ea46da1 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:44.881450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 510/1017] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:44.886582 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3809b7-b095-4a3a-9f54-19b233b41df0 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:44.886983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3809b7-b095-4a3a-9f54-19b233b41df0 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:44.887164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3809b7-b095-4a3a-9f54-19b233b41df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:44.903501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-6b3809b7-b095-4a3a-9f54-19b233b41df0 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:44.904675 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-6b3809b7-b095-4a3a-9f54-19b233b41df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:44.973988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 508/1018] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:44.997646 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61692-b36d-4bc8-bf2d-f3fd4ea46da1 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:45.105409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61692-b36d-4bc8-bf2d-f3fd4ea46da1 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:45.141061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3c678-fde8-4386-b848-16106a831dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:45.141803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3c678-fde8-4386-b848-16106a831dd4 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.142082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3c678-fde8-4386-b848-16106a831dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.142379 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3c678-fde8-4386-b848-16106a831dd4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.152256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3c678-fde8-4386-b848-16106a831dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-81e3c678-fde8-4386-b848-16106a831dd4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:45.160946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:45.162108 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:45.163062 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 511/1019] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:29:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:45.168463 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3809b7-b095-4a3a-9f54-19b233b41df0 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:45.262664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3809b7-b095-4a3a-9f54-19b233b41df0 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:45.274274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9cce9-0134-4269-8f8a-0be6e7abce43 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.274574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9cce9-0134-4269-8f8a-0be6e7abce43 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.274856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9cce9-0134-4269-8f8a-0be6e7abce43 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.276391 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 512/1020] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:45.281201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169839-badd-4969-9a6c-7973381cdf73 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.281495 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169839-badd-4969-9a6c-7973381cdf73 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.281773 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169839-badd-4969-9a6c-7973381cdf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.299042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-8f169839-badd-4969-9a6c-7973381cdf73 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:45.299331 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:45.300004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.300283 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.300559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.301328 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:45.301632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:45.302101 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-8f169839-badd-4969-9a6c-7973381cdf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:45.303002 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:45.310706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 509/1021] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:29:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:45.316189 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:45.375155 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 513/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:45.398766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f169839-badd-4969-9a6c-7973381cdf73 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:45.417313 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1a351-e72f-41f7-8e0c-c6c153a00827 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:45.450136 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:45.450926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.451211 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.451487 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.462524 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:45.463984 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:45.481416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:45.482668 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:45.484239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:45.484239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:45.484239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:45.484239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:45.484239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 510/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:45.488600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:45.495246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f169839-badd-4969-9a6c-7973381cdf73 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:45.528655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66cc15-5097-4d12-b161-a06e6f833c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:45.529492 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66cc15-5097-4d12-b161-a06e6f833c77 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.529778 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66cc15-5097-4d12-b161-a06e6f833c77 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.530056 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66cc15-5097-4d12-b161-a06e6f833c77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.531264 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66cc15-5097-4d12-b161-a06e6f833c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:45.532700 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66cc15-5097-4d12-b161-a06e6f833c77 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:45.535064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 514/1024] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:45.540649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66cc15-5097-4d12-b161-a06e6f833c77 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:45.587048 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80776495-8cd3-4164-8155-7137b67c4d9d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:45.620037 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:45.620707 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.620987 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.621289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.622028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:45.622333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:45.623694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:45.635980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 511/1025] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:45.636442 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66cc15-5097-4d12-b161-a06e6f833c77 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:45.641439 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:45.670580 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ec46-35bb-40f1-b091-bd25feb5b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:45.671172 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ec46-35bb-40f1-b091-bd25feb5b7fb None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.671453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ec46-35bb-40f1-b091-bd25feb5b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.671730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702ec46-35bb-40f1-b091-bd25feb5b7fb None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.672443 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702ec46-35bb-40f1-b091-bd25feb5b7fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:45.673940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702ec46-35bb-40f1-b091-bd25feb5b7fb None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:45.699610 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 515/1026] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:29:45 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:29:45.741796 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda4ab2-c66c-4cc5-b938-409b1ff1b7e5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:45.779940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:45.780760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.781067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.781353 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] PATH_INFO: `/v3/users/nova` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.789599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:45.791068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:45.806545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:45.807628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:45.808651 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 512/1027] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:29:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:45.813890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ec46-35bb-40f1-b091-bd25feb5b7fb None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:45.917439 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702ec46-35bb-40f1-b091-bd25feb5b7fb None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:45.952777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb4e0a-d4b7-48b6-9697-cd5eed92650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:45.953407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb4e0a-d4b7-48b6-9697-cd5eed92650c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:45.953699 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb4e0a-d4b7-48b6-9697-cd5eed92650c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:45.953974 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb4e0a-d4b7-48b6-9697-cd5eed92650c None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:45.954849 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb4e0a-d4b7-48b6-9697-cd5eed92650c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:45.955701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb4e0a-d4b7-48b6-9697-cd5eed92650c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:45.957171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb4e0a-d4b7-48b6-9697-cd5eed92650c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:45.974357 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 516/1028] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:29:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:45.980311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.082775 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c7d5-de5f-4ff9-abc5-4b08eca0ca02 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.117205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90efe6-5770-4c8d-9c45-76ecb4ba6941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.117870 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90efe6-5770-4c8d-9c45-76ecb4ba6941 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.118159 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90efe6-5770-4c8d-9c45-76ecb4ba6941 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.118437 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90efe6-5770-4c8d-9c45-76ecb4ba6941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.183869 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90efe6-5770-4c8d-9c45-76ecb4ba6941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.185361 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90efe6-5770-4c8d-9c45-76ecb4ba6941 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.201751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cc17-b351-48f0-b1ba-33861d06acc1 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.202040 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cc17-b351-48f0-b1ba-33861d06acc1 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.202330 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364cc17-b351-48f0-b1ba-33861d06acc1 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.203966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 517/1029] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:46.209313 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ac00e-6510-4a5a-afe5-e1de629ab057 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.209602 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ac00e-6510-4a5a-afe5-e1de629ab057 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.209886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ac00e-6510-4a5a-afe5-e1de629ab057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.226717 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-b06ac00e-6510-4a5a-afe5-e1de629ab057 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:46.227917 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-b06ac00e-6510-4a5a-afe5-e1de629ab057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:46.285294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 513/1030] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:46.290803 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90efe6-5770-4c8d-9c45-76ecb4ba6941 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.295546 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 518/1031] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:46.318333 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ac00e-6510-4a5a-afe5-e1de629ab057 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.388530 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90efe6-5770-4c8d-9c45-76ecb4ba6941 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.416971 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ac00e-6510-4a5a-afe5-e1de629ab057 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.422751 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.423551 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.424404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.424973 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.434738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.436200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.450291 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.451086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.451372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.451647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] PATH_INFO: `/v3/services/image` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.452313 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.453817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:46.453939 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:46.454901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:46.455877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:46.455877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:46.455877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:46.455877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:46.455877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 514/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:46.460784 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:46.468077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:46.469415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:46.470327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:46.473343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 519/1033] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:46.475607 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.559975 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6d5ea-745f-42ba-8a6a-72965aafef4a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.573588 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0dc8d-031f-478f-8dcc-2a6b811f6f73 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.596319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.597402 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.597922 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.598416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.599382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:46.599899 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.601638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.607302 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.607968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.608255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.608534 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.609659 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] RBAC: Adding query filter params (name=image) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:46.609659 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.610943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.615225 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 515/1034] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:46.617082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 520/1035] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:46.621002 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.621155 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.719686 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7f8c0-9199-43d7-84ac-55dc05737144 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.720219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ea17a-1d47-4371-a885-0194bdd9dab5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.754245 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.755271 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.755712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.756164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.757350 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.758018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.758306 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.758579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.759302 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] RBAC: Adding query filter params (type=image) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:46.759619 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.760992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.762936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.764363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.771222 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 521/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:46.772205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:46.779260 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:46.780207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:46.781219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:46.781219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:46.781219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:46.781219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:46.781219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:46.781219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:46.781219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:46.781219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:46.785777 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 516/1037] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:46.787036 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.873934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb52779-3c60-4a9a-a97b-f121cd329bed None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.888937 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eceef1-e7d4-44e2-baa8-b09db66ad8f6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:46.908440 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.909222 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.909577 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.909931 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.910866 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] RBAC: Adding query filter params (interface=public, service_id=238daa9b109d40219639c3cc5efb3073, region_id=RegionOne) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:46.911255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.912782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.918330 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 522/1038] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/endpoints?service_id=238daa9b109d40219639c3cc5efb3073&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:46.923466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:46.926682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:46.927865 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:46.927865 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:46.928185 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:46.929156 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:46.929580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:46.931118 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:46.944073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 517/1039] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:46.949796 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:47.022616 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d0724-3d10-44c6-8ffb-c9a7dca0bef2 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:47.046961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0fc45-caaf-4aa8-ab17-b431ac76b760 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:47.055644 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea35ad-ee86-465b-9370-a837b4945874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:47.056321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea35ad-ee86-465b-9370-a837b4945874 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:47.056603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea35ad-ee86-465b-9370-a837b4945874 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:47.056878 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea35ad-ee86-465b-9370-a837b4945874 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:47.057606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea35ad-ee86-465b-9370-a837b4945874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:47.059034 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea35ad-ee86-465b-9370-a837b4945874 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:47.066200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 523/1040] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/services => generated 1957 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:29:47.088431 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417433d-6f3a-4e7e-909c-c899c221760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:47.089399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417433d-6f3a-4e7e-909c-c899c221760b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:47.089875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417433d-6f3a-4e7e-909c-c899c221760b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:47.090304 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417433d-6f3a-4e7e-909c-c899c221760b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:47.091337 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417433d-6f3a-4e7e-909c-c899c221760b None admin] RBAC: Adding query filter params (role.id=87f96af882934745afa3b46c6d539918, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=17730dd371184663b2c0eb091cc668a3) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:47.091652 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417433d-6f3a-4e7e-909c-c899c221760b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:47.093148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417433d-6f3a-4e7e-909c-c899c221760b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:47.100831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 518/1041] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:46 2022] GET /identity/v3/role_assignments?user.id=17730dd371184663b2c0eb091cc668a3&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=87f96af882934745afa3b46c6d539918 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:47.997413 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f786-3081-4da3-8737-82091a636155 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:47.997958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f786-3081-4da3-8737-82091a636155 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:47.997958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914f786-3081-4da3-8737-82091a636155 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:47.999525 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 524/1042] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:48.005385 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40665c-8fa7-46ff-abe1-ed9e4a164040 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:48.005688 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40665c-8fa7-46ff-abe1-ed9e4a164040 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:48.005977 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40665c-8fa7-46ff-abe1-ed9e4a164040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:48.024942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-2c40665c-8fa7-46ff-abe1-ed9e4a164040 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:48.026183 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-2c40665c-8fa7-46ff-abe1-ed9e4a164040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:48.091063 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 519/1043] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:48.115590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914f786-3081-4da3-8737-82091a636155 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:48.215381 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914f786-3081-4da3-8737-82091a636155 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:48.252750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2b0d6-d471-4872-a9e7-ae3b54659376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:48.253485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2b0d6-d471-4872-a9e7-ae3b54659376 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:48.253766 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2b0d6-d471-4872-a9e7-ae3b54659376 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:48.254038 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2b0d6-d471-4872-a9e7-ae3b54659376 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:48.255209 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2b0d6-d471-4872-a9e7-ae3b54659376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:48.256636 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2b0d6-d471-4872-a9e7-ae3b54659376 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:48.263044 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 525/1044] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:48.264015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40665c-8fa7-46ff-abe1-ed9e4a164040 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:48.360253 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40665c-8fa7-46ff-abe1-ed9e4a164040 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:48.396564 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17ff77-be38-4f1e-bda3-47a09cf2bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:48.397271 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ff77-be38-4f1e-bda3-47a09cf2bc8c None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:48.397615 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ff77-be38-4f1e-bda3-47a09cf2bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:48.397902 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17ff77-be38-4f1e-bda3-47a09cf2bc8c None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:48.398633 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17ff77-be38-4f1e-bda3-47a09cf2bc8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:48.400082 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17ff77-be38-4f1e-bda3-47a09cf2bc8c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:48.425079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 520/1045] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:29:48 2022] POST /identity/v3/users => generated 285 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:29:49.345877 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01415762-5a08-4837-bc13-1baa151f6960 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.346409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01415762-5a08-4837-bc13-1baa151f6960 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.346493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01415762-5a08-4837-bc13-1baa151f6960 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.347988 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 526/1046] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:49.353096 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619134c-715a-49ed-a0ae-01e11b3cabd9 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.353489 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619134c-715a-49ed-a0ae-01e11b3cabd9 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.353676 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619134c-715a-49ed-a0ae-01e11b3cabd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.353923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149464-720b-4677-bf2d-3e8b055ddeee None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.354214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149464-720b-4677-bf2d-3e8b055ddeee None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.354493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149464-720b-4677-bf2d-3e8b055ddeee None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.355874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 527/1047] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:49.361905 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a13ad1-a729-4b09-803d-f17b25fadf2f None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.362198 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a13ad1-a729-4b09-803d-f17b25fadf2f None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.362486 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a13ad1-a729-4b09-803d-f17b25fadf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.370406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-6619134c-715a-49ed-a0ae-01e11b3cabd9 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:49.371587 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-6619134c-715a-49ed-a0ae-01e11b3cabd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:49.379960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-b9a13ad1-a729-4b09-803d-f17b25fadf2f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:49.381163 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-b9a13ad1-a729-4b09-803d-f17b25fadf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:49.442644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 521/1048] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:49.448581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 528/1049] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:49.465408 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619134c-715a-49ed-a0ae-01e11b3cabd9 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:49.471598 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a13ad1-a729-4b09-803d-f17b25fadf2f None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:49.565331 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619134c-715a-49ed-a0ae-01e11b3cabd9 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:49.567367 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a13ad1-a729-4b09-803d-f17b25fadf2f None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:49.598824 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa64d6e-1121-4558-8793-00eedeb8f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:49.599661 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa64d6e-1121-4558-8793-00eedeb8f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.599942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa64d6e-1121-4558-8793-00eedeb8f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.600220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa64d6e-1121-4558-8793-00eedeb8f2f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.601548 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:49.602366 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.602651 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.602927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] PATH_INFO: `/v3/services/image` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.603663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:49.605018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:49.611779 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa64d6e-1121-4558-8793-00eedeb8f2f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:49.617508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:49.618540 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:49.624663 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 529/1050] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:29:49 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-efa64d6e-1121-4558-8793-00eedeb8f2f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:49.625572 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:49.626478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:49.627926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 522/1051] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:29:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:49.628689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:49.628765 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa64d6e-1121-4558-8793-00eedeb8f2f7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:49.725549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa64d6e-1121-4558-8793-00eedeb8f2f7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:49.727976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984b336-887d-48ec-b265-58e7f1d776e7 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:49.759715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:49.760389 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.760675 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.760955 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.761742 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:49.762054 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:49.763435 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:49.764500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:49.765353 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.765795 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.766219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.767077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] RBAC: Adding query filter params (name=image) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:49.767557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:49.769072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:49.771265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 523/1052] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:29:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:49.776861 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:49.778255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 530/1053] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:49.783768 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:49.874490 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796e752-2342-4c71-9b91-45941c3c2c0f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:49.889244 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cfdfa-1dc0-4ef7-950a-66c7286d465f None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:49.906678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:49.907469 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.907752 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.908065 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.918502 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:49.919929 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:49.922234 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:49.922892 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:49.923175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:49.923451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:49.924693 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] RBAC: Adding query filter params (type=image) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:49.925003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:49.926389 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:49.931233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 531/1054] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:49.936775 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:49.937683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:49.938611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:49.940239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:49.940239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:49.940239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:49.940239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:49.943927 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 524/1055] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:49.944728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:50.037492 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ef1a-1b0c-4f86-9358-a78bf4b91182 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:50.039911 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83784c53-2bde-49e8-83f6-516c454ceec5 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:50.070781 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe89f4-549f-46bb-b52f-a443d87926d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:50.071440 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe89f4-549f-46bb-b52f-a443d87926d4 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.071722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe89f4-549f-46bb-b52f-a443d87926d4 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.071998 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe89f4-549f-46bb-b52f-a443d87926d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.072735 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe89f4-549f-46bb-b52f-a443d87926d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:50.074136 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe89f4-549f-46bb-b52f-a443d87926d4 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:50.076007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:50.076669 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.076953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.077255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.077985 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:50.078298 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:50.079653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:50.096004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 532/1056] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:29:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:29:50.102505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 525/1057] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:50.108387 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe89f4-549f-46bb-b52f-a443d87926d4 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:50.223631 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe89f4-549f-46bb-b52f-a443d87926d4 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:50.260574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:50.261431 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.261718 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.262047 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.271327 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:50.272796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:50.288828 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:50.289940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:50.291070 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 533/1058] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:29:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:50.296038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:50.395146 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850603-ed91-4e3a-b05f-53c75a3f942b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:50.434807 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:50.435505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.435790 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.436065 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.436927 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:50.437281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:50.438824 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:50.461946 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 526/1059] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:29:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:50.463672 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:50.565525 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e7405-d765-4c4e-847d-099232c1a0f9 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:50.598286 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e09dcf-de20-40e4-86e5-f05ed109f084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:50.599085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09dcf-de20-40e4-86e5-f05ed109f084 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.599370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09dcf-de20-40e4-86e5-f05ed109f084 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.599647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e09dcf-de20-40e4-86e5-f05ed109f084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.610363 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e09dcf-de20-40e4-86e5-f05ed109f084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:50.612338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e09dcf-de20-40e4-86e5-f05ed109f084 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-38e09dcf-de20-40e4-86e5-f05ed109f084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:50.629720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:50.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:50.631760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:50.631760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:50.631760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:50.631760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:50.635644 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 534/1060] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:50.637114 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:50.728372 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe725f-832d-4ab3-9b08-521f106cddd3 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.728664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe725f-832d-4ab3-9b08-521f106cddd3 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.728946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe725f-832d-4ab3-9b08-521f106cddd3 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.730456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 535/1061] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:50.733506 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75e33a-5cfc-4231-bede-38f16db10d5a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:50.735590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c6e2-4d5c-4096-afe5-50da612133ce None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.735884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c6e2-4d5c-4096-afe5-50da612133ce None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.736714 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c6e2-4d5c-4096-afe5-50da612133ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.754482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-e5e1c6e2-4d5c-4096-afe5-50da612133ce None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:50.755681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-e5e1c6e2-4d5c-4096-afe5-50da612133ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:50.771401 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:50.772451 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.772915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.773370 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.774266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:50.774726 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:50.776251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:50.788350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 527/1062] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:50.797846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:50.831193 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 536/1063] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:50.858053 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1c6e2-4d5c-4096-afe5-50da612133ce None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:50.906910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0cd93-7bb1-4578-84ea-f8f37c463a19 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:50.942304 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:50.943109 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.943399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.943684 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:50.950448 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:50.951905 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:50.955822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1c6e2-4d5c-4096-afe5-50da612133ce None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:50.966068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:50.967045 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:50.968056 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 528/1064] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:50.973514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:50.994311 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646db23-71db-48f9-97c7-a52a8f3b8f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:50.995355 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646db23-71db-48f9-97c7-a52a8f3b8f8e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:50.995801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646db23-71db-48f9-97c7-a52a8f3b8f8e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:50.996233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646db23-71db-48f9-97c7-a52a8f3b8f8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:51.006295 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646db23-71db-48f9-97c7-a52a8f3b8f8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-b646db23-71db-48f9-97c7-a52a8f3b8f8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:51.014036 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:51.015088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:51.016013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 537/1065] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:51.021397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646db23-71db-48f9-97c7-a52a8f3b8f8e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:51.076968 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692618-6ab1-48cd-9eb4-6eb3b619cc75 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:51.118855 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:51.119560 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:51.119847 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:51.120126 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:51.120919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:51.121259 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:51.122681 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:51.129874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646db23-71db-48f9-97c7-a52a8f3b8f8e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:51.143211 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 529/1066] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:51.144962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:51.163770 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:51.164502 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:51.164787 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:51.165091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:51.165853 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:51.166161 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:51.167523 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:51.180331 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 538/1067] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:51.181219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:51.250637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9344f-6d0a-4a9f-b790-6e3ce82bc9f7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:51.281336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d54155-49ee-48ce-bad1-8154413b81ed None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:51.290386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65bd7d-c052-40e1-8f3c-d2d647fe1932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:51.291275 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65bd7d-c052-40e1-8f3c-d2d647fe1932 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:51.291559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65bd7d-c052-40e1-8f3c-d2d647fe1932 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:51.291835 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65bd7d-c052-40e1-8f3c-d2d647fe1932 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/17730dd371184663b2c0eb091cc668a3/roles/87f96af882934745afa3b46c6d539918` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:51.294127 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65bd7d-c052-40e1-8f3c-d2d647fe1932 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=17730dd371184663b2c0eb091cc668a3, role_id=87f96af882934745afa3b46c6d539918)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:51.295549 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65bd7d-c052-40e1-8f3c-d2d647fe1932 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:51.310395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 530/1068] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:29:51 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/17730dd371184663b2c0eb091cc668a3/roles/87f96af882934745afa3b46c6d539918 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:29:51.321213 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181d4cc-3471-4b62-9aa8-425700f2f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:51.322204 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181d4cc-3471-4b62-9aa8-425700f2f803 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:51.322689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181d4cc-3471-4b62-9aa8-425700f2f803 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:51.322689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181d4cc-3471-4b62-9aa8-425700f2f803 None admin] PATH_INFO: `/v3/users/b17c99c7c490424e97eb4c4221516deb` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:51.339265 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181d4cc-3471-4b62-9aa8-425700f2f803 None admin] RBAC: Authorizing `identity:get_user(user_id=b17c99c7c490424e97eb4c4221516deb)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:51.340762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181d4cc-3471-4b62-9aa8-425700f2f803 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:51.342664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 539/1069] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:29:51 2022] GET /identity/v3/users/b17c99c7c490424e97eb4c4221516deb => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:51.348695 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65bd7d-c052-40e1-8f3c-d2d647fe1932 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:51.460191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65bd7d-c052-40e1-8f3c-d2d647fe1932 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:51.495933 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a4e53-912d-4ff9-b531-1b32d88e91e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:51.496602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4e53-912d-4ff9-b531-1b32d88e91e3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:51.496884 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4e53-912d-4ff9-b531-1b32d88e91e3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:51.497271 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a4e53-912d-4ff9-b531-1b32d88e91e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:51.565545 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a4e53-912d-4ff9-b531-1b32d88e91e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:51.567026 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a4e53-912d-4ff9-b531-1b32d88e91e3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:51.660154 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 531/1070] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:51.665971 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181d4cc-3471-4b62-9aa8-425700f2f803 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:51.761891 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181d4cc-3471-4b62-9aa8-425700f2f803 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:51.794795 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ab894-9438-4a13-966f-72aa1e84ed97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:51.795657 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab894-9438-4a13-966f-72aa1e84ed97 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:51.795940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab894-9438-4a13-966f-72aa1e84ed97 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:51.796215 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab894-9438-4a13-966f-72aa1e84ed97 None admin] PATH_INFO: `/v3/projects/d4b5567c59b3408dbb6b8b0a40808607` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:51.797425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ab894-9438-4a13-966f-72aa1e84ed97 None admin] RBAC: Authorizing `identity:get_project(project_id=d4b5567c59b3408dbb6b8b0a40808607)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:51.799441 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ab894-9438-4a13-966f-72aa1e84ed97 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:51.806250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 540/1071] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:29:51 2022] GET /identity/v3/projects/d4b5567c59b3408dbb6b8b0a40808607 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:51.807465 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a4e53-912d-4ff9-b531-1b32d88e91e3 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:51.903498 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a4e53-912d-4ff9-b531-1b32d88e91e3 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:51.936861 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6bd4-58cc-4291-8654-4d5f1eedc423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:51.937467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6bd4-58cc-4291-8654-4d5f1eedc423 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:51.937753 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6bd4-58cc-4291-8654-4d5f1eedc423 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:51.938030 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6bd4-58cc-4291-8654-4d5f1eedc423 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:51.938868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6bd4-58cc-4291-8654-4d5f1eedc423 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=d4b5567c59b3408dbb6b8b0a40808607, user.id=b17c99c7c490424e97eb4c4221516deb) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:51.939221 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6bd4-58cc-4291-8654-4d5f1eedc423 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:51.940661 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6bd4-58cc-4291-8654-4d5f1eedc423 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:51.953339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 532/1072] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:51 2022] GET /identity/v3/role_assignments?user.id=b17c99c7c490424e97eb4c4221516deb&scope.project.id=d4b5567c59b3408dbb6b8b0a40808607&role.id=58c34600086c469b87bfa39b74dd2607 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:52.391620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca05395-6ee9-426d-905e-36d082d894da None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:52.391804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca05395-6ee9-426d-905e-36d082d894da None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:52.392089 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca05395-6ee9-426d-905e-36d082d894da None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:52.394323 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 541/1073] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:52.399466 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8cf85-cd71-42d8-931d-48dc7635dd0f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:52.399767 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8cf85-cd71-42d8-931d-48dc7635dd0f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:52.400049 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8cf85-cd71-42d8-931d-48dc7635dd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:52.416504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-f7e8cf85-cd71-42d8-931d-48dc7635dd0f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:52.417737 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-f7e8cf85-cd71-42d8-931d-48dc7635dd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:52.488349 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 533/1074] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:52.510193 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca05395-6ee9-426d-905e-36d082d894da None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:52.620052 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca05395-6ee9-426d-905e-36d082d894da None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:52.652301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d8765-ca4f-4095-8c62-0e8383d3efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:52.652959 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8765-ca4f-4095-8c62-0e8383d3efb3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:52.653272 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8765-ca4f-4095-8c62-0e8383d3efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:52.653549 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d8765-ca4f-4095-8c62-0e8383d3efb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:52.719252 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d8765-ca4f-4095-8c62-0e8383d3efb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:52.721300 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d8765-ca4f-4095-8c62-0e8383d3efb3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:52.819399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 542/1075] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:52.821317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8cf85-cd71-42d8-931d-48dc7635dd0f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:52.920198 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8cf85-cd71-42d8-931d-48dc7635dd0f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:52.952209 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed9573-f59d-4bb1-b248-e2346a18c038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:52.953005 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed9573-f59d-4bb1-b248-e2346a18c038 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:52.953322 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed9573-f59d-4bb1-b248-e2346a18c038 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:52.953603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed9573-f59d-4bb1-b248-e2346a18c038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:52.963825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed9573-f59d-4bb1-b248-e2346a18c038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:52.965308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed9573-f59d-4bb1-b248-e2346a18c038 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-6fed9573-f59d-4bb1-b248-e2346a18c038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:52.983342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:52.984303 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:52.985317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:52.985317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:52.985317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:52.985317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:52.989376 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 534/1076] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:52.990621 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d8765-ca4f-4095-8c62-0e8383d3efb3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:53.087889 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d8765-ca4f-4095-8c62-0e8383d3efb3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:53.122972 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a10c4f-baa1-4926-b641-8907d8ba7802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:53.123646 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a10c4f-baa1-4926-b641-8907d8ba7802 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:53.123933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a10c4f-baa1-4926-b641-8907d8ba7802 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:53.124214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a10c4f-baa1-4926-b641-8907d8ba7802 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:53.124945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a10c4f-baa1-4926-b641-8907d8ba7802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:53.125286 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a10c4f-baa1-4926-b641-8907d8ba7802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:53.127301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a10c4f-baa1-4926-b641-8907d8ba7802 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:53.145072 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 543/1077] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:53.573400 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1c8fa-face-4a40-a5ae-0ca5d747d986 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:53.573935 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1c8fa-face-4a40-a5ae-0ca5d747d986 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:53.574043 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1c8fa-face-4a40-a5ae-0ca5d747d986 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:53.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 535/1078] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:53.580743 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a9fb-2457-4f7e-9c2a-5e9a011c13fa None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:53.581132 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a9fb-2457-4f7e-9c2a-5e9a011c13fa None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:53.581482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a9fb-2457-4f7e-9c2a-5e9a011c13fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:53.597760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-5050a9fb-2457-4f7e-9c2a-5e9a011c13fa None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:53.598971 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-5050a9fb-2457-4f7e-9c2a-5e9a011c13fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:53.663975 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 544/1079] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:53.687739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1c8fa-face-4a40-a5ae-0ca5d747d986 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:53.785903 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1c8fa-face-4a40-a5ae-0ca5d747d986 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:53.819552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcd6e5-257f-49a7-b283-8c7b680bda93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:53.820354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd6e5-257f-49a7-b283-8c7b680bda93 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:53.820636 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd6e5-257f-49a7-b283-8c7b680bda93 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:53.820910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcd6e5-257f-49a7-b283-8c7b680bda93 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:53.830712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcd6e5-257f-49a7-b283-8c7b680bda93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-f2bcd6e5-257f-49a7-b283-8c7b680bda93 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:53.838760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:53.839930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:53.840831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 536/1080] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:29:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:53.846465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050a9fb-2457-4f7e-9c2a-5e9a011c13fa None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:53.940666 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050a9fb-2457-4f7e-9c2a-5e9a011c13fa None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:53.972822 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:53.973514 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:53.973797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:53.974068 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:53.974814 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:53.975120 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:53.976505 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:53.984323 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 545/1081] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:29:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:53.990848 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcd6e5-257f-49a7-b283-8c7b680bda93 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.088610 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcd6e5-257f-49a7-b283-8c7b680bda93 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.121400 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1dc28-d742-45aa-9a96-74f26fd8d9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.122185 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1dc28-d742-45aa-9a96-74f26fd8d9f9 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.122467 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1dc28-d742-45aa-9a96-74f26fd8d9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.122743 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1dc28-d742-45aa-9a96-74f26fd8d9f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.132911 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1dc28-d742-45aa-9a96-74f26fd8d9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:54.134374 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1dc28-d742-45aa-9a96-74f26fd8d9f9 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-4da1dc28-d742-45aa-9a96-74f26fd8d9f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:54.150627 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:54.151728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:54.153156 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:54.153156 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:54.153156 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:54.153156 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:54.153156 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 537/1082] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:54.157593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.224504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c505fd-6625-478f-9923-2329569b3bc4 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.224794 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c505fd-6625-478f-9923-2329569b3bc4 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.225111 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c505fd-6625-478f-9923-2329569b3bc4 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.226481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 538/1083] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:29:54.231477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7130-0ffa-42f5-b40a-5679b780ab98 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.232024 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7130-0ffa-42f5-b40a-5679b780ab98 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.232304 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7130-0ffa-42f5-b40a-5679b780ab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.248620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-55af7130-0ffa-42f5-b40a-5679b780ab98 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:54.249837 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-55af7130-0ffa-42f5-b40a-5679b780ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:54.254724 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366cb25-886f-4cd0-8e9b-a2887fc0ef3c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.287157 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.287834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.288101 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.288379 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.289134 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:54.289447 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:54.290815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:54.302437 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 546/1084] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:54.307935 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.314320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 539/1085] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:29:54.337117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af7130-0ffa-42f5-b40a-5679b780ab98 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.401107 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fc4b5-eaab-41e9-a2a1-640215103e38 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.430709 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af7130-0ffa-42f5-b40a-5679b780ab98 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.433415 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.434197 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.434484 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.434762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.441646 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:54.443098 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:29:54.457606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:54.458648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:54.459754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 547/1086] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:54.463196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81c158-9777-473b-92b6-a57a43db959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.464007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81c158-9777-473b-92b6-a57a43db959d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.464293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81c158-9777-473b-92b6-a57a43db959d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.464573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81c158-9777-473b-92b6-a57a43db959d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.464729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.475223 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81c158-9777-473b-92b6-a57a43db959d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-0c81c158-9777-473b-92b6-a57a43db959d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:54.483158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:54.484169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:54.485095 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 540/1087] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:54.490323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81c158-9777-473b-92b6-a57a43db959d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.562740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1556e-931f-4de0-9a82-59bcb4cf7f22 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.589642 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81c158-9777-473b-92b6-a57a43db959d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.595753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.596419 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.596702 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.596980 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.597858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:54.598175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:54.599657 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:54.616450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 548/1088] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:54.622022 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.622909 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.623591 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.623878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.624161 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.624920 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:54.625262 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:54.627196 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:54.639706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 541/1089] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:54.640824 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.716401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eda969-37fe-46c8-ad3c-8ef4c572ab31 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.735583 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8d81a-4c0f-4382-af27-6fdd179b3545 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.749289 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22ddd-f29e-4993-96b5-8b4ecd1c2e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.749944 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22ddd-f29e-4993-96b5-8b4ecd1c2e91 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.750223 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22ddd-f29e-4993-96b5-8b4ecd1c2e91 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.750496 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22ddd-f29e-4993-96b5-8b4ecd1c2e91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.768582 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c04318-48ac-4b38-a60c-db1fe3771125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.769477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c04318-48ac-4b38-a60c-db1fe3771125 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.769763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c04318-48ac-4b38-a60c-db1fe3771125 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.770042 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c04318-48ac-4b38-a60c-db1fe3771125 None admin] PATH_INFO: `/v3/users/b17c99c7c490424e97eb4c4221516deb` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.771286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c04318-48ac-4b38-a60c-db1fe3771125 None admin] RBAC: Authorizing `identity:get_user(user_id=b17c99c7c490424e97eb4c4221516deb)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:54.772734 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c04318-48ac-4b38-a60c-db1fe3771125 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:54.774399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 542/1090] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/users/b17c99c7c490424e97eb4c4221516deb => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:54.779912 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c04318-48ac-4b38-a60c-db1fe3771125 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.815404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22ddd-f29e-4993-96b5-8b4ecd1c2e91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:54.816845 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22ddd-f29e-4993-96b5-8b4ecd1c2e91 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:54.873942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c04318-48ac-4b38-a60c-db1fe3771125 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:54.906379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555542dd-9a12-4e88-8238-d3ad24d11c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:54.907224 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555542dd-9a12-4e88-8238-d3ad24d11c30 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:54.907513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555542dd-9a12-4e88-8238-d3ad24d11c30 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:54.907829 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555542dd-9a12-4e88-8238-d3ad24d11c30 None admin] PATH_INFO: `/v3/projects/d4b5567c59b3408dbb6b8b0a40808607` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:54.908800 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 549/1091] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:29:54.909298 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555542dd-9a12-4e88-8238-d3ad24d11c30 None admin] RBAC: Authorizing `identity:get_project(project_id=d4b5567c59b3408dbb6b8b0a40808607)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:54.910555 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555542dd-9a12-4e88-8238-d3ad24d11c30 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:54.912279 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 543/1092] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/projects/d4b5567c59b3408dbb6b8b0a40808607 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:29:54.914474 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22ddd-f29e-4993-96b5-8b4ecd1c2e91 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:54.917886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555542dd-9a12-4e88-8238-d3ad24d11c30 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:55.007630 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22ddd-f29e-4993-96b5-8b4ecd1c2e91 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:55.012474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555542dd-9a12-4e88-8238-d3ad24d11c30 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:55.038881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:55.039662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:55.039944 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:55.040218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:55.043979 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3fe2f-acd5-4f3f-a424-78b86f3f7109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:55.044854 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3fe2f-acd5-4f3f-a424-78b86f3f7109 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:55.045173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3fe2f-acd5-4f3f-a424-78b86f3f7109 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:55.045459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3fe2f-acd5-4f3f-a424-78b86f3f7109 None admin] PATH_INFO: `/v3/projects/d4b5567c59b3408dbb6b8b0a40808607/users/b17c99c7c490424e97eb4c4221516deb/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:55.047433 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3fe2f-acd5-4f3f-a424-78b86f3f7109 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b5567c59b3408dbb6b8b0a40808607, user_id=b17c99c7c490424e97eb4c4221516deb, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:55.049406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3fe2f-acd5-4f3f-a424-78b86f3f7109 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:55.050493 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:55.051906 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:55.061966 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 544/1093] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:29:54 2022] PUT /identity/v3/projects/d4b5567c59b3408dbb6b8b0a40808607/users/b17c99c7c490424e97eb4c4221516deb/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:29:55.069869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:55.070938 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:55.072342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:29:55.072342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:29:55.072342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:29:55.072342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:55.072342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 550/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:55.077276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3fe2f-acd5-4f3f-a424-78b86f3f7109 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:55.181309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3fe2f-acd5-4f3f-a424-78b86f3f7109 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:55.215703 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:55.216386 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:55.216673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:55.216953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:55.217737 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:55.218048 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:55.219433 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:55.232131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 545/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:55.238360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:55.333414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd7bbd-cd51-41ff-90ad-32fbaff15a6d None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:55.369448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:55.370248 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:55.370530 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:55.370804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:55.377746 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:55.379179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:29:55.392512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:55.393655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:29:55.394911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 551/1096] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:29:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:29:55.401531 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:55.495798 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b3f44-e5db-4a1b-8e93-85491875b9a6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:55.527265 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6d5ae-e903-4c35-a9ee-174ff3e6b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:55.527918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6d5ae-e903-4c35-a9ee-174ff3e6b139 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:55.528200 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6d5ae-e903-4c35-a9ee-174ff3e6b139 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:55.528472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6d5ae-e903-4c35-a9ee-174ff3e6b139 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:55.529296 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6d5ae-e903-4c35-a9ee-174ff3e6b139 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:55.529603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6d5ae-e903-4c35-a9ee-174ff3e6b139 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:55.531015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6d5ae-e903-4c35-a9ee-174ff3e6b139 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:55.542631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 546/1097] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:29:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:55.549467 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:55.648167 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171ff80-f393-4406-9198-93f7293b7314 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:55.682143 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18623494-d024-431e-b1b8-d7a38344b6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:55.682797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18623494-d024-431e-b1b8-d7a38344b6db None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:55.683084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18623494-d024-431e-b1b8-d7a38344b6db None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:55.683361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18623494-d024-431e-b1b8-d7a38344b6db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:55.684199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18623494-d024-431e-b1b8-d7a38344b6db None admin] RBAC: Adding query filter params (role.id=87f96af882934745afa3b46c6d539918, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=17730dd371184663b2c0eb091cc668a3) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:55.684517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18623494-d024-431e-b1b8-d7a38344b6db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:55.685983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18623494-d024-431e-b1b8-d7a38344b6db None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:55.691720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 552/1098] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:55 2022] GET /identity/v3/role_assignments?user.id=17730dd371184663b2c0eb091cc668a3&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=87f96af882934745afa3b46c6d539918 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:57.299198 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea961281-026b-4f59-bc73-6cebdad8431f None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:57.299783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea961281-026b-4f59-bc73-6cebdad8431f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:57.299783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea961281-026b-4f59-bc73-6cebdad8431f None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:57.301145 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 547/1099] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:29:57.306424 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3100ccf-cc8e-4f7d-b2ce-d394071c3746 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:57.306873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3100ccf-cc8e-4f7d-b2ce-d394071c3746 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:57.306978 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3100ccf-cc8e-4f7d-b2ce-d394071c3746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:57.323371 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-b3100ccf-cc8e-4f7d-b2ce-d394071c3746 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:29:57.324566 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-b3100ccf-cc8e-4f7d-b2ce-d394071c3746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:29:57.392554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 553/1100] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:29:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:29:57.415855 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea961281-026b-4f59-bc73-6cebdad8431f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:57.512853 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea961281-026b-4f59-bc73-6cebdad8431f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:57.545516 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce5865-5769-484c-b6d5-721fb8d25b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:57.546297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce5865-5769-484c-b6d5-721fb8d25b07 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:57.546588 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce5865-5769-484c-b6d5-721fb8d25b07 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:57.546870 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce5865-5769-484c-b6d5-721fb8d25b07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:57.556625 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce5865-5769-484c-b6d5-721fb8d25b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-39ce5865-5769-484c-b6d5-721fb8d25b07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:29:57.564799 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:29:57.566073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:29:57.567058 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 548/1101] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:29:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:29:57.571739 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3100ccf-cc8e-4f7d-b2ce-d394071c3746 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:57.665563 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3100ccf-cc8e-4f7d-b2ce-d394071c3746 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:57.697575 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:57.698236 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:57.698522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:57.698802 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:57.699554 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:57.699871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:57.701270 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:57.709779 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 554/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:29:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:57.715687 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce5865-5769-484c-b6d5-721fb8d25b07 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:57.811033 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce5865-5769-484c-b6d5-721fb8d25b07 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:57.843113 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54292d65-b797-4a0b-8fa4-b6e84b28d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:57.843964 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54292d65-b797-4a0b-8fa4-b6e84b28d8ba None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:57.844249 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54292d65-b797-4a0b-8fa4-b6e84b28d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:57.844526 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54292d65-b797-4a0b-8fa4-b6e84b28d8ba None admin] PATH_INFO: `/v3/users/b17c99c7c490424e97eb4c4221516deb` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:57.845755 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54292d65-b797-4a0b-8fa4-b6e84b28d8ba None admin] RBAC: Authorizing `identity:get_user(user_id=b17c99c7c490424e97eb4c4221516deb)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:57.847188 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54292d65-b797-4a0b-8fa4-b6e84b28d8ba None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:57.848779 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 549/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:29:57 2022] GET /identity/v3/users/b17c99c7c490424e97eb4c4221516deb => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:57.854469 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:57.952916 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96bbe4-6f6d-4e25-8617-73dfcf4989cb None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:57.984620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf69eb-316b-4de6-90fa-4324e4f238fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:57.985284 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf69eb-316b-4de6-90fa-4324e4f238fc None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:57.985567 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf69eb-316b-4de6-90fa-4324e4f238fc None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:57.986343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf69eb-316b-4de6-90fa-4324e4f238fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:58.049768 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf69eb-316b-4de6-90fa-4324e4f238fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:58.051211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf69eb-316b-4de6-90fa-4324e4f238fc None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:58.143608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 555/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:29:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:29:58.149464 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54292d65-b797-4a0b-8fa4-b6e84b28d8ba None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:58.243172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54292d65-b797-4a0b-8fa4-b6e84b28d8ba None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:58.274511 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1284a2-e812-4bfe-abf1-b12649ac5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:58.275360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1284a2-e812-4bfe-abf1-b12649ac5639 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:58.275645 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1284a2-e812-4bfe-abf1-b12649ac5639 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:58.275925 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1284a2-e812-4bfe-abf1-b12649ac5639 None admin] PATH_INFO: `/v3/projects/d4b5567c59b3408dbb6b8b0a40808607` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:58.277136 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1284a2-e812-4bfe-abf1-b12649ac5639 None admin] RBAC: Authorizing `identity:get_project(project_id=d4b5567c59b3408dbb6b8b0a40808607)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:58.278557 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1284a2-e812-4bfe-abf1-b12649ac5639 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:58.280210 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 550/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:29:58 2022] GET /identity/v3/projects/d4b5567c59b3408dbb6b8b0a40808607 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:29:58.286091 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf69eb-316b-4de6-90fa-4324e4f238fc None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:29:58.380009 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf69eb-316b-4de6-90fa-4324e4f238fc None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:29:58.412212 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea5fa9-571d-4f42-b53a-829602470080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:29:58.412864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea5fa9-571d-4f42-b53a-829602470080 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:29:58.413267 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea5fa9-571d-4f42-b53a-829602470080 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:29:58.413634 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea5fa9-571d-4f42-b53a-829602470080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:29:58.414669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea5fa9-571d-4f42-b53a-829602470080 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=d4b5567c59b3408dbb6b8b0a40808607, user.id=b17c99c7c490424e97eb4c4221516deb) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:29:58.415017 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea5fa9-571d-4f42-b53a-829602470080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:29:58.416750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea5fa9-571d-4f42-b53a-829602470080 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:29:58.422457 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 556/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:29:58 2022] GET /identity/v3/role_assignments?user.id=b17c99c7c490424e97eb4c4221516deb&scope.project.id=d4b5567c59b3408dbb6b8b0a40808607&role.id=58c34600086c469b87bfa39b74dd2607 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:00.669481 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6493f-4a51-4f1e-a688-90a5d862ea12 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:00.670092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6493f-4a51-4f1e-a688-90a5d862ea12 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:00.670092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6493f-4a51-4f1e-a688-90a5d862ea12 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:00.671641 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 551/1107] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:00.678369 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484f7ed-2c7d-49fa-ac6f-1a0743791811 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:00.679029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484f7ed-2c7d-49fa-ac6f-1a0743791811 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:00.679140 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484f7ed-2c7d-49fa-ac6f-1a0743791811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:00.696587 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-2484f7ed-2c7d-49fa-ac6f-1a0743791811 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:00.697820 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-2484f7ed-2c7d-49fa-ac6f-1a0743791811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:00.761399 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 557/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:30:00.785978 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6493f-4a51-4f1e-a688-90a5d862ea12 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:00.881577 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6493f-4a51-4f1e-a688-90a5d862ea12 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:00.913123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22200eda-54c6-495d-a435-fdac39712a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:00.913900 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200eda-54c6-495d-a435-fdac39712a8d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:00.914180 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200eda-54c6-495d-a435-fdac39712a8d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:00.914454 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200eda-54c6-495d-a435-fdac39712a8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:00.915622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22200eda-54c6-495d-a435-fdac39712a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:00.917057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22200eda-54c6-495d-a435-fdac39712a8d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:00.918638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 552/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:00.924924 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484f7ed-2c7d-49fa-ac6f-1a0743791811 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:01.018184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484f7ed-2c7d-49fa-ac6f-1a0743791811 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:01.050258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42891a9-2543-452b-b658-dc84fff7b415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:01.050918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42891a9-2543-452b-b658-dc84fff7b415 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:01.051203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42891a9-2543-452b-b658-dc84fff7b415 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:01.051482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42891a9-2543-452b-b658-dc84fff7b415 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:01.052201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42891a9-2543-452b-b658-dc84fff7b415 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:01.053671 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42891a9-2543-452b-b658-dc84fff7b415 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:01.072019 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 558/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:30:00 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:30:03.307299 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5afc3d-1e9c-4d4b-8b2a-1a6ba79b7bfc None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:03.307850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5afc3d-1e9c-4d4b-8b2a-1a6ba79b7bfc None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:03.307850 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5afc3d-1e9c-4d4b-8b2a-1a6ba79b7bfc None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:03.309234 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 553/1111] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:03.315201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518a038-67ba-48e2-be9b-be69e4d9a79e None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:03.315606 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518a038-67ba-48e2-be9b-be69e4d9a79e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:03.315760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518a038-67ba-48e2-be9b-be69e4d9a79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:03.332024 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-f518a038-67ba-48e2-be9b-be69e4d9a79e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:03.333239 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-f518a038-67ba-48e2-be9b-be69e4d9a79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:03.398003 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 559/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:03.422211 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5afc3d-1e9c-4d4b-8b2a-1a6ba79b7bfc None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:03.518186 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5afc3d-1e9c-4d4b-8b2a-1a6ba79b7bfc None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:03.550054 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5a487-8736-435e-9f86-e7b9387635cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:03.550888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5a487-8736-435e-9f86-e7b9387635cb None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:03.551171 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5a487-8736-435e-9f86-e7b9387635cb None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:03.551447 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5a487-8736-435e-9f86-e7b9387635cb None admin] PATH_INFO: `/v3/roles/02f09975329342eead41c01b150325c0` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:03.552904 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5a487-8736-435e-9f86-e7b9387635cb None admin] RBAC: Authorizing `identity:get_role(role_id=02f09975329342eead41c01b150325c0)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:03.554295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5a487-8736-435e-9f86-e7b9387635cb None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:03.555638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 554/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:03 2022] GET /identity/v3/roles/02f09975329342eead41c01b150325c0 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:03.561453 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518a038-67ba-48e2-be9b-be69e4d9a79e None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:03.654460 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518a038-67ba-48e2-be9b-be69e4d9a79e None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:03.686918 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082033e-c83b-44d2-b6db-6ed2c8c91968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:03.687776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082033e-c83b-44d2-b6db-6ed2c8c91968 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:03.688065 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082033e-c83b-44d2-b6db-6ed2c8c91968 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:03.688342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082033e-c83b-44d2-b6db-6ed2c8c91968 None admin] PATH_INFO: `/v3/users/9ed4cd0578264e31bfedd2282aa07069` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:03.700754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082033e-c83b-44d2-b6db-6ed2c8c91968 None admin] RBAC: Authorizing `identity:get_user(user_id=9ed4cd0578264e31bfedd2282aa07069)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:03.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082033e-c83b-44d2-b6db-6ed2c8c91968 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:03.703934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 560/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:03 2022] GET /identity/v3/users/9ed4cd0578264e31bfedd2282aa07069 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:03.710293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5a487-8736-435e-9f86-e7b9387635cb None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:03.810258 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5a487-8736-435e-9f86-e7b9387635cb None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:03.841676 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a6266-36c0-4e0b-98c2-8133dab1c376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:03.842317 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a6266-36c0-4e0b-98c2-8133dab1c376 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:03.842608 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a6266-36c0-4e0b-98c2-8133dab1c376 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:03.842888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a6266-36c0-4e0b-98c2-8133dab1c376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:03.905965 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a6266-36c0-4e0b-98c2-8133dab1c376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:03.907351 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a6266-36c0-4e0b-98c2-8133dab1c376 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:04.000756 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 555/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:30:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:30:04.006715 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082033e-c83b-44d2-b6db-6ed2c8c91968 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:04.102479 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082033e-c83b-44d2-b6db-6ed2c8c91968 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:04.136511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01343102-5fdb-4947-84ba-350ab02fd04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:04.137401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01343102-5fdb-4947-84ba-350ab02fd04e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:04.137690 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01343102-5fdb-4947-84ba-350ab02fd04e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:04.137970 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01343102-5fdb-4947-84ba-350ab02fd04e None admin] PATH_INFO: `/v3/projects/d4b5567c59b3408dbb6b8b0a40808607` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:04.139167 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01343102-5fdb-4947-84ba-350ab02fd04e None admin] RBAC: Authorizing `identity:get_project(project_id=d4b5567c59b3408dbb6b8b0a40808607)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:04.140611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01343102-5fdb-4947-84ba-350ab02fd04e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:04.142904 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 561/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:04 2022] GET /identity/v3/projects/d4b5567c59b3408dbb6b8b0a40808607 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:04.149148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a6266-36c0-4e0b-98c2-8133dab1c376 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:04.243425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a6266-36c0-4e0b-98c2-8133dab1c376 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:04.275368 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9b806-4bb7-457d-8441-b1358d160b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:04.276009 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9b806-4bb7-457d-8441-b1358d160b4d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:04.276296 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9b806-4bb7-457d-8441-b1358d160b4d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:04.276574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9b806-4bb7-457d-8441-b1358d160b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:04.277429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9b806-4bb7-457d-8441-b1358d160b4d None admin] RBAC: Adding query filter params (role.id=02f09975329342eead41c01b150325c0, scope.project.id=d4b5567c59b3408dbb6b8b0a40808607, user.id=9ed4cd0578264e31bfedd2282aa07069) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:04.277744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9b806-4bb7-457d-8441-b1358d160b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:04.279160 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9b806-4bb7-457d-8441-b1358d160b4d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:04.284911 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 556/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:30:04 2022] GET /identity/v3/role_assignments?user.id=9ed4cd0578264e31bfedd2282aa07069&scope.project.id=d4b5567c59b3408dbb6b8b0a40808607&role.id=02f09975329342eead41c01b150325c0 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:06.474416 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57183ce-5394-4597-a6d6-e37fe22ce1f4 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:06.474961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57183ce-5394-4597-a6d6-e37fe22ce1f4 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:06.474961 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57183ce-5394-4597-a6d6-e37fe22ce1f4 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:06.476342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 562/1118] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:06.481506 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074c7b6-a09d-49d1-bcad-2aed49c01847 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:06.482152 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074c7b6-a09d-49d1-bcad-2aed49c01847 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:06.482332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074c7b6-a09d-49d1-bcad-2aed49c01847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:06.499208 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-e074c7b6-a09d-49d1-bcad-2aed49c01847 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:06.500404 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-e074c7b6-a09d-49d1-bcad-2aed49c01847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:06.567262 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 557/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:06.589565 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57183ce-5394-4597-a6d6-e37fe22ce1f4 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:06.685539 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57183ce-5394-4597-a6d6-e37fe22ce1f4 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:06.717013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934336b-1f3c-49bb-9a0b-1ad8db36bd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:06.717888 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934336b-1f3c-49bb-9a0b-1ad8db36bd43 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:06.718170 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934336b-1f3c-49bb-9a0b-1ad8db36bd43 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:06.718443 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934336b-1f3c-49bb-9a0b-1ad8db36bd43 None admin] PATH_INFO: `/v3/roles/02f09975329342eead41c01b150325c0` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:06.719917 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934336b-1f3c-49bb-9a0b-1ad8db36bd43 None admin] RBAC: Authorizing `identity:get_role(role_id=02f09975329342eead41c01b150325c0)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:06.721819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934336b-1f3c-49bb-9a0b-1ad8db36bd43 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:06.723156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 563/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:06 2022] GET /identity/v3/roles/02f09975329342eead41c01b150325c0 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:06.728744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074c7b6-a09d-49d1-bcad-2aed49c01847 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:06.826055 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074c7b6-a09d-49d1-bcad-2aed49c01847 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:06.858254 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8948a-20db-4b57-beaf-c109ff1380df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:06.859123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8948a-20db-4b57-beaf-c109ff1380df None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:06.859405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8948a-20db-4b57-beaf-c109ff1380df None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:06.859681 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8948a-20db-4b57-beaf-c109ff1380df None admin] PATH_INFO: `/v3/users/9ed4cd0578264e31bfedd2282aa07069` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:06.861449 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8948a-20db-4b57-beaf-c109ff1380df None admin] RBAC: Authorizing `identity:get_user(user_id=9ed4cd0578264e31bfedd2282aa07069)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:06.862886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8948a-20db-4b57-beaf-c109ff1380df None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:06.864499 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 558/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:06 2022] GET /identity/v3/users/9ed4cd0578264e31bfedd2282aa07069 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:06.870299 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934336b-1f3c-49bb-9a0b-1ad8db36bd43 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:06.963267 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934336b-1f3c-49bb-9a0b-1ad8db36bd43 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:06.995900 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427b99c-aa37-4cbd-b9df-1df38fd94a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:06.996679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427b99c-aa37-4cbd-b9df-1df38fd94a0f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:06.996958 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427b99c-aa37-4cbd-b9df-1df38fd94a0f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:06.997255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427b99c-aa37-4cbd-b9df-1df38fd94a0f None admin] PATH_INFO: `/v3/projects/d4b5567c59b3408dbb6b8b0a40808607` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:06.998421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427b99c-aa37-4cbd-b9df-1df38fd94a0f None admin] RBAC: Authorizing `identity:get_project(project_id=d4b5567c59b3408dbb6b8b0a40808607)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:06.999834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427b99c-aa37-4cbd-b9df-1df38fd94a0f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:07.001721 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 564/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:06 2022] GET /identity/v3/projects/d4b5567c59b3408dbb6b8b0a40808607 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:07.007272 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8948a-20db-4b57-beaf-c109ff1380df None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:07.101796 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8948a-20db-4b57-beaf-c109ff1380df None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:07.150153 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8830659-3502-4226-bc1d-7875f77fbeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:07.151061 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830659-3502-4226-bc1d-7875f77fbeb7 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:07.151349 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830659-3502-4226-bc1d-7875f77fbeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:07.151632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830659-3502-4226-bc1d-7875f77fbeb7 None admin] PATH_INFO: `/v3/projects/d4b5567c59b3408dbb6b8b0a40808607/users/9ed4cd0578264e31bfedd2282aa07069/roles/02f09975329342eead41c01b150325c0` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:07.153956 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8830659-3502-4226-bc1d-7875f77fbeb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4b5567c59b3408dbb6b8b0a40808607, user_id=9ed4cd0578264e31bfedd2282aa07069, role_id=02f09975329342eead41c01b150325c0)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:07.155617 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8830659-3502-4226-bc1d-7875f77fbeb7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:07.170110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 559/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:30:07 2022] PUT /identity/v3/projects/d4b5567c59b3408dbb6b8b0a40808607/users/9ed4cd0578264e31bfedd2282aa07069/roles/02f09975329342eead41c01b150325c0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:30:09.417547 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9bf43-a24e-45f1-ad60-5b87992fb22b None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:09.418118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9bf43-a24e-45f1-ad60-5b87992fb22b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:09.418118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9bf43-a24e-45f1-ad60-5b87992fb22b None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:09.419461 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 565/1124] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:09.424914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614166-3448-4de0-9696-7d6af2b0f448 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:09.425332 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614166-3448-4de0-9696-7d6af2b0f448 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:09.425517 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2614166-3448-4de0-9696-7d6af2b0f448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:09.441726 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-e2614166-3448-4de0-9696-7d6af2b0f448 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:09.442908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-e2614166-3448-4de0-9696-7d6af2b0f448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:09.506001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 560/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:09.529069 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9bf43-a24e-45f1-ad60-5b87992fb22b None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:09.624647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9bf43-a24e-45f1-ad60-5b87992fb22b None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:09.657155 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd395c-13e0-4212-b4d7-38b4155dc9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:09.658027 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd395c-13e0-4212-b4d7-38b4155dc9de None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:09.658308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd395c-13e0-4212-b4d7-38b4155dc9de None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:09.658582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd395c-13e0-4212-b4d7-38b4155dc9de None admin] PATH_INFO: `/v3/roles/02f09975329342eead41c01b150325c0` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:09.660057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd395c-13e0-4212-b4d7-38b4155dc9de None admin] RBAC: Authorizing `identity:get_role(role_id=02f09975329342eead41c01b150325c0)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:09.661462 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd395c-13e0-4212-b4d7-38b4155dc9de None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:09.662811 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 566/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:09 2022] GET /identity/v3/roles/02f09975329342eead41c01b150325c0 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:09.668561 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2614166-3448-4de0-9696-7d6af2b0f448 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:09.762917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2614166-3448-4de0-9696-7d6af2b0f448 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:09.794805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef73813-0253-49d3-aacd-4a90d432995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:09.795664 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef73813-0253-49d3-aacd-4a90d432995a None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:09.795950 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef73813-0253-49d3-aacd-4a90d432995a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:09.796227 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef73813-0253-49d3-aacd-4a90d432995a None admin] PATH_INFO: `/v3/users/9ed4cd0578264e31bfedd2282aa07069` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:09.797429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef73813-0253-49d3-aacd-4a90d432995a None admin] RBAC: Authorizing `identity:get_user(user_id=9ed4cd0578264e31bfedd2282aa07069)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:09.799004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef73813-0253-49d3-aacd-4a90d432995a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:09.800599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 561/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:09 2022] GET /identity/v3/users/9ed4cd0578264e31bfedd2282aa07069 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:09.806406 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd395c-13e0-4212-b4d7-38b4155dc9de None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:09.899903 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd395c-13e0-4212-b4d7-38b4155dc9de None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:09.930857 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7899e-d336-4786-857b-55eaff2a922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:09.931426 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7899e-d336-4786-857b-55eaff2a922b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:09.931706 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7899e-d336-4786-857b-55eaff2a922b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:09.931980 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7899e-d336-4786-857b-55eaff2a922b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:09.994915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7899e-d336-4786-857b-55eaff2a922b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:09.996345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7899e-d336-4786-857b-55eaff2a922b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:10.087966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 567/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:30:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:30:10.093881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef73813-0253-49d3-aacd-4a90d432995a None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:10.186681 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef73813-0253-49d3-aacd-4a90d432995a None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:10.219090 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab4ca-4c76-4446-b9d0-3d032c2f3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:10.220552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab4ca-4c76-4446-b9d0-3d032c2f3076 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:10.220841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab4ca-4c76-4446-b9d0-3d032c2f3076 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:10.221191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab4ca-4c76-4446-b9d0-3d032c2f3076 None admin] PATH_INFO: `/v3/projects/d4b5567c59b3408dbb6b8b0a40808607` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:10.222354 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ab4ca-4c76-4446-b9d0-3d032c2f3076 None admin] RBAC: Authorizing `identity:get_project(project_id=d4b5567c59b3408dbb6b8b0a40808607)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:10.223923 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ab4ca-4c76-4446-b9d0-3d032c2f3076 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:10.225649 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 562/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:10 2022] GET /identity/v3/projects/d4b5567c59b3408dbb6b8b0a40808607 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:10.231966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7899e-d336-4786-857b-55eaff2a922b None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:10.324777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7899e-d336-4786-857b-55eaff2a922b None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:10.357167 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35caeada-3327-4a29-bcde-4f343dc100c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:10.357810 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caeada-3327-4a29-bcde-4f343dc100c9 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:10.358092 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caeada-3327-4a29-bcde-4f343dc100c9 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:10.358370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caeada-3327-4a29-bcde-4f343dc100c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:10.359202 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35caeada-3327-4a29-bcde-4f343dc100c9 None admin] RBAC: Adding query filter params (role.id=02f09975329342eead41c01b150325c0, scope.project.id=d4b5567c59b3408dbb6b8b0a40808607, user.id=9ed4cd0578264e31bfedd2282aa07069) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:10.359517 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35caeada-3327-4a29-bcde-4f343dc100c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:10.360943 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35caeada-3327-4a29-bcde-4f343dc100c9 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:10.371027 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 568/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:30:10 2022] GET /identity/v3/role_assignments?user.id=9ed4cd0578264e31bfedd2282aa07069&scope.project.id=d4b5567c59b3408dbb6b8b0a40808607&role.id=02f09975329342eead41c01b150325c0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:12.648173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe697bbd-8456-4ea2-abbe-304bc10e7c16 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:12.648744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe697bbd-8456-4ea2-abbe-304bc10e7c16 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:12.648744 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe697bbd-8456-4ea2-abbe-304bc10e7c16 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:12.650123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 563/1131] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:12.655414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3473f-d1ff-4a9c-b51b-f41f1a87302f None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:12.655836 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3473f-d1ff-4a9c-b51b-f41f1a87302f None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:12.655973 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3473f-d1ff-4a9c-b51b-f41f1a87302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:12.672648 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-4bf3473f-d1ff-4a9c-b51b-f41f1a87302f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:12.674436 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-4bf3473f-d1ff-4a9c-b51b-f41f1a87302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:12.737444 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 569/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:30:12.760483 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe697bbd-8456-4ea2-abbe-304bc10e7c16 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:12.855805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe697bbd-8456-4ea2-abbe-304bc10e7c16 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:12.887734 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2b2df-0cf0-4603-81f0-ec91ee91e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:12.888504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b2df-0cf0-4603-81f0-ec91ee91e205 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:12.888788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b2df-0cf0-4603-81f0-ec91ee91e205 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:12.889088 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b2df-0cf0-4603-81f0-ec91ee91e205 None admin] PATH_INFO: `/v3/domains/default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:12.890255 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2b2df-0cf0-4603-81f0-ec91ee91e205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:12.891704 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2b2df-0cf0-4603-81f0-ec91ee91e205 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:12.893308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 564/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:30:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:12.898948 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3473f-d1ff-4a9c-b51b-f41f1a87302f None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:12.992173 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3473f-d1ff-4a9c-b51b-f41f1a87302f None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:13.023782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a589dc2-c903-4414-8020-b7a08934acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:13.024421 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a589dc2-c903-4414-8020-b7a08934acc3 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:13.024706 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a589dc2-c903-4414-8020-b7a08934acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:13.024979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a589dc2-c903-4414-8020-b7a08934acc3 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:13.025710 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a589dc2-c903-4414-8020-b7a08934acc3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:13.027123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a589dc2-c903-4414-8020-b7a08934acc3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:13.051272 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 570/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:30:12 2022] POST /identity/v3/projects => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:30:15.321396 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f57f8-211f-4824-b370-6e54cac68b1c None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:15.321949 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f57f8-211f-4824-b370-6e54cac68b1c None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:15.321949 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f57f8-211f-4824-b370-6e54cac68b1c None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:15.323292 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 565/1135] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:15.328658 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0b65-610a-4f91-b050-42ac0c6550ad None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:15.329620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0b65-610a-4f91-b050-42ac0c6550ad None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:15.329791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0b65-610a-4f91-b050-42ac0c6550ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:15.346698 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-ffab0b65-610a-4f91-b050-42ac0c6550ad None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:15.347879 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-ffab0b65-610a-4f91-b050-42ac0c6550ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:15.411357 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 571/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:15.439277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f57f8-211f-4824-b370-6e54cac68b1c None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:15.539826 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f57f8-211f-4824-b370-6e54cac68b1c None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:15.571938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2b700-168f-4e6b-90cc-08e74d9e37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:15.572719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2b700-168f-4e6b-90cc-08e74d9e37fc None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:15.573003 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2b700-168f-4e6b-90cc-08e74d9e37fc None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:15.573318 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2b700-168f-4e6b-90cc-08e74d9e37fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:15.574476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2b700-168f-4e6b-90cc-08e74d9e37fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:15.575901 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2b700-168f-4e6b-90cc-08e74d9e37fc None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:15.577516 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 566/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:30:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:15.583852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0b65-610a-4f91-b050-42ac0c6550ad None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:15.677723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0b65-610a-4f91-b050-42ac0c6550ad None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:15.710836 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1ee18-6f06-4101-9519-48dfa6b0c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:15.711500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1ee18-6f06-4101-9519-48dfa6b0c700 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:15.711788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1ee18-6f06-4101-9519-48dfa6b0c700 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:15.712067 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1ee18-6f06-4101-9519-48dfa6b0c700 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:15.712800 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1ee18-6f06-4101-9519-48dfa6b0c700 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:15.714274 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1ee18-6f06-4101-9519-48dfa6b0c700 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:15.732435 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 572/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:30:15 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:30:17.968958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d522e-f671-41b7-82ec-3b02b9a0f332 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:17.969556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d522e-f671-41b7-82ec-3b02b9a0f332 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:17.969556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d522e-f671-41b7-82ec-3b02b9a0f332 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:17.970929 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 567/1139] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:17.976104 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfbaa5-db14-47f8-babc-b80a34348f8f None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:17.976553 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfbaa5-db14-47f8-babc-b80a34348f8f None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:17.976661 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfbaa5-db14-47f8-babc-b80a34348f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:17.993340 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-33dfbaa5-db14-47f8-babc-b80a34348f8f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:17.994530 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-33dfbaa5-db14-47f8-babc-b80a34348f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:18.057229 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 573/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:18.081283 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d522e-f671-41b7-82ec-3b02b9a0f332 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:18.177561 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d522e-f671-41b7-82ec-3b02b9a0f332 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:18.210014 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64781086-df63-4eeb-9ea8-bc9ea3b99aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:18.210801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64781086-df63-4eeb-9ea8-bc9ea3b99aff None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:18.211088 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64781086-df63-4eeb-9ea8-bc9ea3b99aff None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:18.211368 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64781086-df63-4eeb-9ea8-bc9ea3b99aff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:18.221219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64781086-df63-4eeb-9ea8-bc9ea3b99aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-64781086-df63-4eeb-9ea8-bc9ea3b99aff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:30:18.228719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:30:18.229983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:30:18.231108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 568/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:30:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:30:18.236967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfbaa5-db14-47f8-babc-b80a34348f8f None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:18.330771 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfbaa5-db14-47f8-babc-b80a34348f8f None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:18.364047 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:18.364725 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:18.365009 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:18.365315 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:18.366080 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:18.366393 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:18.367772 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:18.376345 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 574/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:30:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:18.382738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64781086-df63-4eeb-9ea8-bc9ea3b99aff None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:18.482552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64781086-df63-4eeb-9ea8-bc9ea3b99aff None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:18.515787 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1576f-5946-48e1-b4a9-253ea9003837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:18.516698 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1576f-5946-48e1-b4a9-253ea9003837 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:18.516983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1576f-5946-48e1-b4a9-253ea9003837 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:18.517291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1576f-5946-48e1-b4a9-253ea9003837 None admin] PATH_INFO: `/v3/users/9fa06fb31e664d709a1b13d0f4f96c89` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:18.531205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1576f-5946-48e1-b4a9-253ea9003837 None admin] RBAC: Authorizing `identity:get_user(user_id=9fa06fb31e664d709a1b13d0f4f96c89)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:18.532704 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1576f-5946-48e1-b4a9-253ea9003837 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:18.539072 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 569/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:18 2022] GET /identity/v3/users/9fa06fb31e664d709a1b13d0f4f96c89 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:18.540338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:18.639344 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553c20b-47b6-46b9-a09a-eab6f441bbdd None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:18.670945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0774266-412a-4f3b-ba48-7491ef128419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:18.671585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0774266-412a-4f3b-ba48-7491ef128419 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:18.671869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0774266-412a-4f3b-ba48-7491ef128419 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:18.672148 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0774266-412a-4f3b-ba48-7491ef128419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:18.739395 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0774266-412a-4f3b-ba48-7491ef128419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:18.740803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0774266-412a-4f3b-ba48-7491ef128419 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:18.832385 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 575/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:30:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:30:18.838283 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1576f-5946-48e1-b4a9-253ea9003837 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:18.932943 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1576f-5946-48e1-b4a9-253ea9003837 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:18.965327 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c440b71-96d1-4394-9197-1f6891551e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:18.966173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c440b71-96d1-4394-9197-1f6891551e24 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:18.966454 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c440b71-96d1-4394-9197-1f6891551e24 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:18.966729 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c440b71-96d1-4394-9197-1f6891551e24 None admin] PATH_INFO: `/v3/projects/8e085573a6294bf8b89907e7be655c1f` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:18.967905 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c440b71-96d1-4394-9197-1f6891551e24 None admin] RBAC: Authorizing `identity:get_project(project_id=8e085573a6294bf8b89907e7be655c1f)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:18.969369 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c440b71-96d1-4394-9197-1f6891551e24 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:18.971631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 570/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:18 2022] GET /identity/v3/projects/8e085573a6294bf8b89907e7be655c1f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:18.977608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0774266-412a-4f3b-ba48-7491ef128419 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:19.069712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0774266-412a-4f3b-ba48-7491ef128419 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:19.102420 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35cf99-aedd-4a47-b646-9221085cc010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:19.103061 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35cf99-aedd-4a47-b646-9221085cc010 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:19.103342 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35cf99-aedd-4a47-b646-9221085cc010 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:19.103614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35cf99-aedd-4a47-b646-9221085cc010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:19.104438 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35cf99-aedd-4a47-b646-9221085cc010 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=8e085573a6294bf8b89907e7be655c1f, user.id=9fa06fb31e664d709a1b13d0f4f96c89) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:19.104747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35cf99-aedd-4a47-b646-9221085cc010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:19.106194 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35cf99-aedd-4a47-b646-9221085cc010 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:19.111722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 576/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:30:18 2022] GET /identity/v3/role_assignments?user.id=9fa06fb31e664d709a1b13d0f4f96c89&scope.project.id=8e085573a6294bf8b89907e7be655c1f&role.id=58c34600086c469b87bfa39b74dd2607 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:21.322880 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc79c3f-a1b5-40d3-ac39-d119beabdec8 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:21.323432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc79c3f-a1b5-40d3-ac39-d119beabdec8 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:21.323432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc79c3f-a1b5-40d3-ac39-d119beabdec8 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:21.324792 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 571/1147] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:21.330552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfd956-5305-4153-9fc8-8b3610c7d950 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:21.330978 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfd956-5305-4153-9fc8-8b3610c7d950 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:21.331101 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfd956-5305-4153-9fc8-8b3610c7d950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:21.347763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-76cfd956-5305-4153-9fc8-8b3610c7d950 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:21.348982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-76cfd956-5305-4153-9fc8-8b3610c7d950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:21.413466 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 577/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:21.435757 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc79c3f-a1b5-40d3-ac39-d119beabdec8 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:21.544948 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc79c3f-a1b5-40d3-ac39-d119beabdec8 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:21.580171 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd230c2-6759-45d0-aa13-e0573f763a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:21.580962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd230c2-6759-45d0-aa13-e0573f763a86 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:21.581288 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd230c2-6759-45d0-aa13-e0573f763a86 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:21.581573 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd230c2-6759-45d0-aa13-e0573f763a86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:21.591288 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd230c2-6759-45d0-aa13-e0573f763a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-7dd230c2-6759-45d0-aa13-e0573f763a86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:30:21.598878 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:30:21.599908 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:30:21.600885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 572/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:30:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:30:21.606211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfd956-5305-4153-9fc8-8b3610c7d950 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:21.714657 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfd956-5305-4153-9fc8-8b3610c7d950 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:21.748083 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:21.748759 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:21.749066 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:21.749360 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:21.750127 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:21.750439 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:21.752333 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:21.760723 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 578/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:30:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:21.767529 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd230c2-6759-45d0-aa13-e0573f763a86 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:21.864815 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd230c2-6759-45d0-aa13-e0573f763a86 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:21.897631 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae593-27c7-4cd5-be3f-e9f112c50f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:21.898500 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae593-27c7-4cd5-be3f-e9f112c50f22 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:21.898786 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae593-27c7-4cd5-be3f-e9f112c50f22 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:21.899063 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae593-27c7-4cd5-be3f-e9f112c50f22 None admin] PATH_INFO: `/v3/users/9fa06fb31e664d709a1b13d0f4f96c89` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:21.900245 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dae593-27c7-4cd5-be3f-e9f112c50f22 None admin] RBAC: Authorizing `identity:get_user(user_id=9fa06fb31e664d709a1b13d0f4f96c89)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:21.901742 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dae593-27c7-4cd5-be3f-e9f112c50f22 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:21.903370 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 573/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:21 2022] GET /identity/v3/users/9fa06fb31e664d709a1b13d0f4f96c89 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:21.909430 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:22.005552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7232847-5a20-4be7-a158-4cfcd43b2f73 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:22.036825 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317ba79-967b-4f09-b484-7038e1cca21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:22.037693 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ba79-967b-4f09-b484-7038e1cca21f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:22.037982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ba79-967b-4f09-b484-7038e1cca21f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:22.038257 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317ba79-967b-4f09-b484-7038e1cca21f None admin] PATH_INFO: `/v3/projects/8e085573a6294bf8b89907e7be655c1f` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:22.039432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317ba79-967b-4f09-b484-7038e1cca21f None admin] RBAC: Authorizing `identity:get_project(project_id=8e085573a6294bf8b89907e7be655c1f)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:22.040897 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317ba79-967b-4f09-b484-7038e1cca21f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:22.042616 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 579/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:21 2022] GET /identity/v3/projects/8e085573a6294bf8b89907e7be655c1f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:22.048989 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae593-27c7-4cd5-be3f-e9f112c50f22 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:22.145164 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae593-27c7-4cd5-be3f-e9f112c50f22 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:22.176965 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f36ca2-a360-49ab-a12b-a19aa60d0a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:22.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f36ca2-a360-49ab-a12b-a19aa60d0a50 None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:22.178157 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f36ca2-a360-49ab-a12b-a19aa60d0a50 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:22.178439 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f36ca2-a360-49ab-a12b-a19aa60d0a50 None admin] PATH_INFO: `/v3/projects/8e085573a6294bf8b89907e7be655c1f/users/9fa06fb31e664d709a1b13d0f4f96c89/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:22.180396 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f36ca2-a360-49ab-a12b-a19aa60d0a50 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e085573a6294bf8b89907e7be655c1f, user_id=9fa06fb31e664d709a1b13d0f4f96c89, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:22.181865 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f36ca2-a360-49ab-a12b-a19aa60d0a50 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:22.194842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 574/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:30:22 2022] PUT /identity/v3/projects/8e085573a6294bf8b89907e7be655c1f/users/9fa06fb31e664d709a1b13d0f4f96c89/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:30:24.427785 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e50d4-fd7e-4076-b3ff-ef85f64ae755 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:24.428324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e50d4-fd7e-4076-b3ff-ef85f64ae755 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:24.428423 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e50d4-fd7e-4076-b3ff-ef85f64ae755 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:24.429791 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 580/1154] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:24.435323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f2aba-b4ee-4542-9a97-03e537035cf7 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:24.435756 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f2aba-b4ee-4542-9a97-03e537035cf7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:24.435869 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f2aba-b4ee-4542-9a97-03e537035cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:24.452721 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-492f2aba-b4ee-4542-9a97-03e537035cf7 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:24.453919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-492f2aba-b4ee-4542-9a97-03e537035cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:24.516451 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 575/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:24.538921 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e50d4-fd7e-4076-b3ff-ef85f64ae755 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:24.635718 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e50d4-fd7e-4076-b3ff-ef85f64ae755 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:24.668887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5458a1-55f3-4bfd-a439-ff5745329746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:24.669702 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5458a1-55f3-4bfd-a439-ff5745329746 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:24.669988 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5458a1-55f3-4bfd-a439-ff5745329746 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:24.670262 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5458a1-55f3-4bfd-a439-ff5745329746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:24.681100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5458a1-55f3-4bfd-a439-ff5745329746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-2f5458a1-55f3-4bfd-a439-ff5745329746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:30:24.688593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:30:24.689767 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:30:24.690884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 581/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:30:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:30:24.695946 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f2aba-b4ee-4542-9a97-03e537035cf7 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:24.789339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f2aba-b4ee-4542-9a97-03e537035cf7 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:24.825858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:24.826514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:24.826797 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:24.827070 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:24.827811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:24.828114 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:24.829624 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:24.837509 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 576/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:30:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:24.843324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5458a1-55f3-4bfd-a439-ff5745329746 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:24.942511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5458a1-55f3-4bfd-a439-ff5745329746 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:24.975726 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5df55-b471-46f1-8d2d-a08788d33e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:24.976584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5df55-b471-46f1-8d2d-a08788d33e34 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:24.976866 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5df55-b471-46f1-8d2d-a08788d33e34 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:24.977168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5df55-b471-46f1-8d2d-a08788d33e34 None admin] PATH_INFO: `/v3/users/9fa06fb31e664d709a1b13d0f4f96c89` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:24.978361 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5df55-b471-46f1-8d2d-a08788d33e34 None admin] RBAC: Authorizing `identity:get_user(user_id=9fa06fb31e664d709a1b13d0f4f96c89)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:24.979805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5df55-b471-46f1-8d2d-a08788d33e34 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:24.981711 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 582/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:24 2022] GET /identity/v3/users/9fa06fb31e664d709a1b13d0f4f96c89 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:24.987496 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:25.081883 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df491c-e9da-4527-8e15-333ee3d172b4 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:25.113927 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e5846-81f3-4d0c-b9ec-65436bff925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:25.114567 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e5846-81f3-4d0c-b9ec-65436bff925b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:25.114846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e5846-81f3-4d0c-b9ec-65436bff925b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:25.115118 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e5846-81f3-4d0c-b9ec-65436bff925b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:25.178182 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e5846-81f3-4d0c-b9ec-65436bff925b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:25.179617 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e5846-81f3-4d0c-b9ec-65436bff925b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:25.270963 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 577/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:30:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:30:25.276815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5df55-b471-46f1-8d2d-a08788d33e34 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:25.370899 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5df55-b471-46f1-8d2d-a08788d33e34 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:25.403210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de316d-b18d-4be7-9c45-f503f13498a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:25.404053 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de316d-b18d-4be7-9c45-f503f13498a1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:25.404333 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de316d-b18d-4be7-9c45-f503f13498a1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:25.404605 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de316d-b18d-4be7-9c45-f503f13498a1 None admin] PATH_INFO: `/v3/projects/8e085573a6294bf8b89907e7be655c1f` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:25.405803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de316d-b18d-4be7-9c45-f503f13498a1 None admin] RBAC: Authorizing `identity:get_project(project_id=8e085573a6294bf8b89907e7be655c1f)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:25.407250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de316d-b18d-4be7-9c45-f503f13498a1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:25.408912 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 583/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:25 2022] GET /identity/v3/projects/8e085573a6294bf8b89907e7be655c1f => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:25.414976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e5846-81f3-4d0c-b9ec-65436bff925b None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:25.508074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e5846-81f3-4d0c-b9ec-65436bff925b None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:25.540502 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6285ba-ce4e-43b0-b1bb-96f6086c9e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:25.541114 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6285ba-ce4e-43b0-b1bb-96f6086c9e5f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:25.541400 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6285ba-ce4e-43b0-b1bb-96f6086c9e5f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:25.541677 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6285ba-ce4e-43b0-b1bb-96f6086c9e5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:25.542504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6285ba-ce4e-43b0-b1bb-96f6086c9e5f None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=8e085573a6294bf8b89907e7be655c1f, user.id=9fa06fb31e664d709a1b13d0f4f96c89) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:25.542814 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6285ba-ce4e-43b0-b1bb-96f6086c9e5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:25.544232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6285ba-ce4e-43b0-b1bb-96f6086c9e5f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:25.549917 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 578/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:30:25 2022] GET /identity/v3/role_assignments?user.id=9fa06fb31e664d709a1b13d0f4f96c89&scope.project.id=8e085573a6294bf8b89907e7be655c1f&role.id=58c34600086c469b87bfa39b74dd2607 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:27.830123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188054a-7b3f-45fe-a223-b8b68080ba32 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:27.830745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188054a-7b3f-45fe-a223-b8b68080ba32 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:27.830745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188054a-7b3f-45fe-a223-b8b68080ba32 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:27.832048 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 584/1162] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:27.837315 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547cf4-f0a5-4511-821c-32c53119a943 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:27.837761 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547cf4-f0a5-4511-821c-32c53119a943 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:27.837918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547cf4-f0a5-4511-821c-32c53119a943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:27.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-6b547cf4-f0a5-4511-821c-32c53119a943 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:27.855742 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-6b547cf4-f0a5-4511-821c-32c53119a943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:27.922684 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 579/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:27.945419 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188054a-7b3f-45fe-a223-b8b68080ba32 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:28.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188054a-7b3f-45fe-a223-b8b68080ba32 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:28.079954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e63832-3829-4cf2-8bab-568a8d904e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:28.080645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e63832-3829-4cf2-8bab-568a8d904e7a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:28.080880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e63832-3829-4cf2-8bab-568a8d904e7a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:28.081193 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e63832-3829-4cf2-8bab-568a8d904e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:28.145298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e63832-3829-4cf2-8bab-568a8d904e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:28.146872 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e63832-3829-4cf2-8bab-568a8d904e7a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:28.239881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 585/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:30:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:30:28.245872 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b547cf4-f0a5-4511-821c-32c53119a943 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:28.340251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b547cf4-f0a5-4511-821c-32c53119a943 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:28.373924 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423f0ba-eba7-416f-a232-3708eda57e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:28.374720 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f0ba-eba7-416f-a232-3708eda57e9f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:28.375003 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f0ba-eba7-416f-a232-3708eda57e9f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:28.375278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f0ba-eba7-416f-a232-3708eda57e9f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:28.382235 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423f0ba-eba7-416f-a232-3708eda57e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:28.383683 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423f0ba-eba7-416f-a232-3708eda57e9f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-0423f0ba-eba7-416f-a232-3708eda57e9f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:30:28.397772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:30:28.399177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:30:28.400484 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 580/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:30:28 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:30:28.404911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e63832-3829-4cf2-8bab-568a8d904e7a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:28.496963 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e63832-3829-4cf2-8bab-568a8d904e7a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:28.529994 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c27de-4396-4f78-aafd-eb3f1bfff040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:28.530633 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c27de-4396-4f78-aafd-eb3f1bfff040 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:28.530912 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c27de-4396-4f78-aafd-eb3f1bfff040 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:28.531184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c27de-4396-4f78-aafd-eb3f1bfff040 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:28.531896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c27de-4396-4f78-aafd-eb3f1bfff040 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:28.532238 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c27de-4396-4f78-aafd-eb3f1bfff040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:28.533631 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c27de-4396-4f78-aafd-eb3f1bfff040 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:28.540404 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 586/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:30:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:30.724040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f8fe8-d736-412a-8e3a-5f96fd2d2bb7 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:30.724602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f8fe8-d736-412a-8e3a-5f96fd2d2bb7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:30.724602 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f8fe8-d736-412a-8e3a-5f96fd2d2bb7 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:30.726004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 581/1167] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:30.734849 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02f540-a0bb-4e24-9980-485e06285b72 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:30.735446 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02f540-a0bb-4e24-9980-485e06285b72 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:30.735869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02f540-a0bb-4e24-9980-485e06285b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:30.755599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-db02f540-a0bb-4e24-9980-485e06285b72 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:30.756805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-db02f540-a0bb-4e24-9980-485e06285b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:30.820412 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 587/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:30.843472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f8fe8-d736-412a-8e3a-5f96fd2d2bb7 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:30.941539 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f8fe8-d736-412a-8e3a-5f96fd2d2bb7 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:30.973366 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93fbfd-cfdc-48ad-8b7b-f4b662f55997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:30.974023 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93fbfd-cfdc-48ad-8b7b-f4b662f55997 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:30.974347 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93fbfd-cfdc-48ad-8b7b-f4b662f55997 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:30.974585 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93fbfd-cfdc-48ad-8b7b-f4b662f55997 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:30.975338 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93fbfd-cfdc-48ad-8b7b-f4b662f55997 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:30.976710 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93fbfd-cfdc-48ad-8b7b-f4b662f55997 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:30.991355 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 582/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Mon May 9 15:30:30 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:30:33.228770 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f83381-8bd9-412f-9c35-3b173e1eaa46 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:33.229336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f83381-8bd9-412f-9c35-3b173e1eaa46 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:33.229336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f83381-8bd9-412f-9c35-3b173e1eaa46 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:33.230714 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 588/1170] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:33.236906 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6881f-f7ae-4841-870b-a03d6cd4a45b None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:33.237348 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6881f-f7ae-4841-870b-a03d6cd4a45b None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:33.237508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6881f-f7ae-4841-870b-a03d6cd4a45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:33.254270 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-a9a6881f-f7ae-4841-870b-a03d6cd4a45b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:33.255453 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-a9a6881f-f7ae-4841-870b-a03d6cd4a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:33.318118 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 583/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:33.340768 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f83381-8bd9-412f-9c35-3b173e1eaa46 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:33.434806 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f83381-8bd9-412f-9c35-3b173e1eaa46 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:33.466482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a2ee2-0d03-442d-83db-210b8064d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:33.467324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2ee2-0d03-442d-83db-210b8064d843 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:33.467600 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2ee2-0d03-442d-83db-210b8064d843 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:33.467871 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a2ee2-0d03-442d-83db-210b8064d843 None admin] PATH_INFO: `/v3/domains/0a6dfe005ddc47e0baa89c111c4e7464` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:33.477885 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a2ee2-0d03-442d-83db-210b8064d843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0a6dfe005ddc47e0baa89c111c4e7464)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:33.479322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a2ee2-0d03-442d-83db-210b8064d843 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:33.480922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 589/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 9 15:30:33 2022] GET /identity/v3/domains/0a6dfe005ddc47e0baa89c111c4e7464 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:33.487520 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6881f-f7ae-4841-870b-a03d6cd4a45b None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:33.587785 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6881f-f7ae-4841-870b-a03d6cd4a45b None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:33.620584 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb97d0-d7d9-4777-93cb-6df29aacce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:33.621267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb97d0-d7d9-4777-93cb-6df29aacce78 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:33.621556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb97d0-d7d9-4777-93cb-6df29aacce78 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:33.621835 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb97d0-d7d9-4777-93cb-6df29aacce78 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:33.622638 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb97d0-d7d9-4777-93cb-6df29aacce78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:33.624089 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb97d0-d7d9-4777-93cb-6df29aacce78 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:33.647817 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 584/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:30:33 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:30:35.894064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137c340-b170-47d6-ad40-0cd160f25033 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:35.894625 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137c340-b170-47d6-ad40-0cd160f25033 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:35.894625 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137c340-b170-47d6-ad40-0cd160f25033 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:35.895996 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 590/1174] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:35.901475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17e7e6-a0c7-409c-bdbc-27ffc8645b6b None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:35.901928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17e7e6-a0c7-409c-bdbc-27ffc8645b6b None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:35.902127 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17e7e6-a0c7-409c-bdbc-27ffc8645b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:35.919568 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-9f17e7e6-a0c7-409c-bdbc-27ffc8645b6b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:35.920763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-9f17e7e6-a0c7-409c-bdbc-27ffc8645b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:35.984841 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 585/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:36.009539 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137c340-b170-47d6-ad40-0cd160f25033 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:36.105688 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137c340-b170-47d6-ad40-0cd160f25033 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:36.137578 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34436b46-f955-42d8-8994-95a8726c098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:36.138432 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436b46-f955-42d8-8994-95a8726c098b None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:36.138708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436b46-f955-42d8-8994-95a8726c098b None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:36.138979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34436b46-f955-42d8-8994-95a8726c098b None admin] PATH_INFO: `/v3/domains/0a6dfe005ddc47e0baa89c111c4e7464` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:36.140131 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34436b46-f955-42d8-8994-95a8726c098b None admin] RBAC: Authorizing `identity:get_domain(domain_id=0a6dfe005ddc47e0baa89c111c4e7464)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:36.142253 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34436b46-f955-42d8-8994-95a8726c098b None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:36.143856 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 591/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 9 15:30:36 2022] GET /identity/v3/domains/0a6dfe005ddc47e0baa89c111c4e7464 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:36.149696 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17e7e6-a0c7-409c-bdbc-27ffc8645b6b None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:36.241621 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17e7e6-a0c7-409c-bdbc-27ffc8645b6b None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:36.273339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e66d9f-abab-4d4e-931a-dc8785d235ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:36.273993 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e66d9f-abab-4d4e-931a-dc8785d235ea None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:36.274276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e66d9f-abab-4d4e-931a-dc8785d235ea None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:36.274548 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e66d9f-abab-4d4e-931a-dc8785d235ea None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:36.275257 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e66d9f-abab-4d4e-931a-dc8785d235ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:36.276678 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e66d9f-abab-4d4e-931a-dc8785d235ea None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:36.295461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 586/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:30:36 2022] POST /identity/v3/users => generated 311 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:30:38.580457 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431348-b836-4fef-9cca-81c3f7807700 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:38.581001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431348-b836-4fef-9cca-81c3f7807700 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:38.581001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89431348-b836-4fef-9cca-81c3f7807700 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:38.582413 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 592/1178] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:38.587788 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d2b5-ec04-4f4c-b2e0-319e41d14c1b None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:38.588215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d2b5-ec04-4f4c-b2e0-319e41d14c1b None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:38.588356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3d2b5-ec04-4f4c-b2e0-319e41d14c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:38.605986 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-08e3d2b5-ec04-4f4c-b2e0-319e41d14c1b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:38.607416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-08e3d2b5-ec04-4f4c-b2e0-319e41d14c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:38.674364 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 587/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:38.698103 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431348-b836-4fef-9cca-81c3f7807700 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:38.798652 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89431348-b836-4fef-9cca-81c3f7807700 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:38.832020 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc878b5-c8e6-4a61-a0fd-17aa1877515e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:38.833210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc878b5-c8e6-4a61-a0fd-17aa1877515e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:38.833625 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc878b5-c8e6-4a61-a0fd-17aa1877515e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:38.834042 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc878b5-c8e6-4a61-a0fd-17aa1877515e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:38.849235 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc878b5-c8e6-4a61-a0fd-17aa1877515e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-fcc878b5-c8e6-4a61-a0fd-17aa1877515e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:30:38.858321 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:30:38.859400 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:30:38.860397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 593/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:30:38.867575 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d2b5-ec04-4f4c-b2e0-319e41d14c1b None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:38.961300 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3d2b5-ec04-4f4c-b2e0-319e41d14c1b None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:38.993802 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:38.994471 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:38.994750 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:38.995024 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:38.995773 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:38.996077 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:38.997474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:39.005853 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 588/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:39.012035 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc878b5-c8e6-4a61-a0fd-17aa1877515e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:39.109217 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc878b5-c8e6-4a61-a0fd-17aa1877515e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:39.141705 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc82054-eea5-4e8f-9812-894f91d1bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:39.142565 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc82054-eea5-4e8f-9812-894f91d1bd09 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:39.142842 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc82054-eea5-4e8f-9812-894f91d1bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:39.143114 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc82054-eea5-4e8f-9812-894f91d1bd09 None admin] PATH_INFO: `/v3/users/d6145fd5a11b493d934a55dafe044a3d` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:39.156296 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc82054-eea5-4e8f-9812-894f91d1bd09 None admin] RBAC: Authorizing `identity:get_user(user_id=d6145fd5a11b493d934a55dafe044a3d)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:39.158322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc82054-eea5-4e8f-9812-894f91d1bd09 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:39.159954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 594/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:39 2022] GET /identity/v3/users/d6145fd5a11b493d934a55dafe044a3d => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:39.165783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:39.260248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d55d4-7dc3-45eb-9ad9-3b4d94136b29 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:39.292490 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5ab0-5179-44d5-a912-3951c9622dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:39.293173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5ab0-5179-44d5-a912-3951c9622dff None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:39.293461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5ab0-5179-44d5-a912-3951c9622dff None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:39.293741 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b5ab0-5179-44d5-a912-3951c9622dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:39.357177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b5ab0-5179-44d5-a912-3951c9622dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:39.358565 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b5ab0-5179-44d5-a912-3951c9622dff None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:39.453273 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 589/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:30:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:30:39.459821 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc82054-eea5-4e8f-9812-894f91d1bd09 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:39.555497 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc82054-eea5-4e8f-9812-894f91d1bd09 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:39.589131 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b04e98-a877-4900-b727-d1d454b30004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:39.590022 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04e98-a877-4900-b727-d1d454b30004 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:39.590307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04e98-a877-4900-b727-d1d454b30004 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:39.590581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04e98-a877-4900-b727-d1d454b30004 None admin] PATH_INFO: `/v3/projects/f9016a548ee34e14b91d58e5379543bc` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:39.591771 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b04e98-a877-4900-b727-d1d454b30004 None admin] RBAC: Authorizing `identity:get_project(project_id=f9016a548ee34e14b91d58e5379543bc)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:39.593292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b04e98-a877-4900-b727-d1d454b30004 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:39.594989 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 595/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:39 2022] GET /identity/v3/projects/f9016a548ee34e14b91d58e5379543bc => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:39.601077 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5ab0-5179-44d5-a912-3951c9622dff None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:39.694824 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b5ab0-5179-44d5-a912-3951c9622dff None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:39.727399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c70841-3ff6-4329-b05d-e2c610aaa91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:39.728048 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70841-3ff6-4329-b05d-e2c610aaa91b None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:39.728336 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70841-3ff6-4329-b05d-e2c610aaa91b None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:39.728620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70841-3ff6-4329-b05d-e2c610aaa91b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:39.729491 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c70841-3ff6-4329-b05d-e2c610aaa91b None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=f9016a548ee34e14b91d58e5379543bc, user.id=d6145fd5a11b493d934a55dafe044a3d) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:39.729817 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c70841-3ff6-4329-b05d-e2c610aaa91b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:39.731520 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c70841-3ff6-4329-b05d-e2c610aaa91b None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:39.736746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 590/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:30:39 2022] GET /identity/v3/role_assignments?user.id=d6145fd5a11b493d934a55dafe044a3d&scope.project.id=f9016a548ee34e14b91d58e5379543bc&role.id=58c34600086c469b87bfa39b74dd2607 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:42.035198 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204da8c9-8625-4cda-95a4-083b54b28dab None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:42.035751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204da8c9-8625-4cda-95a4-083b54b28dab None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:42.035751 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204da8c9-8625-4cda-95a4-083b54b28dab None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:42.037166 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 596/1186] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:42.042514 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a75a8-d4f2-427b-a6be-00896334d41d None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:42.042922 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a75a8-d4f2-427b-a6be-00896334d41d None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:42.043077 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a75a8-d4f2-427b-a6be-00896334d41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:42.059930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-cc0a75a8-d4f2-427b-a6be-00896334d41d None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:42.061131 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-cc0a75a8-d4f2-427b-a6be-00896334d41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:42.124482 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 591/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:42.147078 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204da8c9-8625-4cda-95a4-083b54b28dab None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:42.246608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204da8c9-8625-4cda-95a4-083b54b28dab None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:42.278797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa58d4-df3a-41f5-83e7-0691012d5041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:42.279579 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa58d4-df3a-41f5-83e7-0691012d5041 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:42.279858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa58d4-df3a-41f5-83e7-0691012d5041 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:42.280132 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa58d4-df3a-41f5-83e7-0691012d5041 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:42.290862 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa58d4-df3a-41f5-83e7-0691012d5041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-72fa58d4-df3a-41f5-83e7-0691012d5041 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:30:42.298324 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:30:42.299510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:30:42.300712 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 597/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:30:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:30:42.305801 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a75a8-d4f2-427b-a6be-00896334d41d None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:42.400091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a75a8-d4f2-427b-a6be-00896334d41d None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:42.432111 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:42.432766 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:42.433071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:42.433356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:42.434099 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:42.434409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:42.435784 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:42.443451 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 592/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:30:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:42.449348 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa58d4-df3a-41f5-83e7-0691012d5041 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:42.550669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa58d4-df3a-41f5-83e7-0691012d5041 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:42.583811 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7e760-d254-4b30-850f-44968a0b445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:42.584680 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e760-d254-4b30-850f-44968a0b445a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:42.584965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e760-d254-4b30-850f-44968a0b445a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:42.585269 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7e760-d254-4b30-850f-44968a0b445a None admin] PATH_INFO: `/v3/users/d6145fd5a11b493d934a55dafe044a3d` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:42.586473 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7e760-d254-4b30-850f-44968a0b445a None admin] RBAC: Authorizing `identity:get_user(user_id=d6145fd5a11b493d934a55dafe044a3d)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:42.587901 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7e760-d254-4b30-850f-44968a0b445a None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:42.589570 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 598/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:42 2022] GET /identity/v3/users/d6145fd5a11b493d934a55dafe044a3d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:42.595407 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:42.696367 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762528e9-7597-4f07-b1ac-8372a400fe6c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:42.729094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d72c09-1fd9-4823-b5c3-e6996b4ad543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:42.729957 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72c09-1fd9-4823-b5c3-e6996b4ad543 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:42.730241 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72c09-1fd9-4823-b5c3-e6996b4ad543 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:42.730515 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72c09-1fd9-4823-b5c3-e6996b4ad543 None admin] PATH_INFO: `/v3/projects/f9016a548ee34e14b91d58e5379543bc` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:42.731942 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d72c09-1fd9-4823-b5c3-e6996b4ad543 None admin] RBAC: Authorizing `identity:get_project(project_id=f9016a548ee34e14b91d58e5379543bc)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:42.733403 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d72c09-1fd9-4823-b5c3-e6996b4ad543 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:42.735112 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 593/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:42 2022] GET /identity/v3/projects/f9016a548ee34e14b91d58e5379543bc => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:42.741009 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7e760-d254-4b30-850f-44968a0b445a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:42.846050 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7e760-d254-4b30-850f-44968a0b445a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:42.879678 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9f2a5-f56c-4098-8549-e8e0727f0118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:42.880510 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9f2a5-f56c-4098-8549-e8e0727f0118 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:42.880792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9f2a5-f56c-4098-8549-e8e0727f0118 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:42.881095 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9f2a5-f56c-4098-8549-e8e0727f0118 None admin] PATH_INFO: `/v3/projects/f9016a548ee34e14b91d58e5379543bc/users/d6145fd5a11b493d934a55dafe044a3d/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:42.883104 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9f2a5-f56c-4098-8549-e8e0727f0118 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9016a548ee34e14b91d58e5379543bc, user_id=d6145fd5a11b493d934a55dafe044a3d, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:42.884548 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9f2a5-f56c-4098-8549-e8e0727f0118 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:42.897983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 599/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:30:42 2022] PUT /identity/v3/projects/f9016a548ee34e14b91d58e5379543bc/users/d6145fd5a11b493d934a55dafe044a3d/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:30:45.173593 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4131-ecb9-454f-a24a-d79155f45772 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:45.174125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4131-ecb9-454f-a24a-d79155f45772 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:45.174125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4131-ecb9-454f-a24a-d79155f45772 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:45.175516 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 594/1193] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:30:45.180913 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9c762-619c-4837-9a0a-1103835743da None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:45.181373 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9c762-619c-4837-9a0a-1103835743da None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:45.181534 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9c762-619c-4837-9a0a-1103835743da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:45.198475 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-e8e9c762-619c-4837-9a0a-1103835743da None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:30:45.199675 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-e8e9c762-619c-4837-9a0a-1103835743da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:30:45.266776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 600/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:30:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:30:45.290544 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4131-ecb9-454f-a24a-d79155f45772 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:45.391685 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4131-ecb9-454f-a24a-d79155f45772 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:45.425623 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f839c-47f5-4b3f-a007-91b751c35063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:45.426466 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f839c-47f5-4b3f-a007-91b751c35063 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:45.426747 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f839c-47f5-4b3f-a007-91b751c35063 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:45.427027 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f839c-47f5-4b3f-a007-91b751c35063 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:45.436913 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f839c-47f5-4b3f-a007-91b751c35063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-4b1f839c-47f5-4b3f-a007-91b751c35063 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:30:45.444501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:30:45.445851 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:30:45.446995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 595/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:30:45.451885 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9c762-619c-4837-9a0a-1103835743da None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:45.557384 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9c762-619c-4837-9a0a-1103835743da None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:45.597595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:45.598270 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:45.598549 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:45.598824 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:45.599584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:45.599889 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:45.601305 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:45.612275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 601/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:30:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:45.618121 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f839c-47f5-4b3f-a007-91b751c35063 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:45.722564 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f839c-47f5-4b3f-a007-91b751c35063 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:45.756289 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5373-9a63-474e-8eb8-6ee440f7fe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:45.757193 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5373-9a63-474e-8eb8-6ee440f7fe6f None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:45.757470 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5373-9a63-474e-8eb8-6ee440f7fe6f None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:45.757743 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab5373-9a63-474e-8eb8-6ee440f7fe6f None admin] PATH_INFO: `/v3/users/d6145fd5a11b493d934a55dafe044a3d` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:45.758928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab5373-9a63-474e-8eb8-6ee440f7fe6f None admin] RBAC: Authorizing `identity:get_user(user_id=d6145fd5a11b493d934a55dafe044a3d)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:45.760363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab5373-9a63-474e-8eb8-6ee440f7fe6f None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:45.762030 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 596/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:30:45 2022] GET /identity/v3/users/d6145fd5a11b493d934a55dafe044a3d => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:45.768401 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:45.869518 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c45553-f490-4e79-a7b4-e5ceda9eea79 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:45.914575 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc8fdc-67a2-4b8c-911e-fcb56ec796da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:45.915231 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc8fdc-67a2-4b8c-911e-fcb56ec796da None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:45.915514 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc8fdc-67a2-4b8c-911e-fcb56ec796da None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:45.915789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc8fdc-67a2-4b8c-911e-fcb56ec796da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:45.982965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc8fdc-67a2-4b8c-911e-fcb56ec796da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:45.984364 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc8fdc-67a2-4b8c-911e-fcb56ec796da None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:46.080763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 602/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:30:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:30:46.086740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5373-9a63-474e-8eb8-6ee440f7fe6f None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:46.186084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab5373-9a63-474e-8eb8-6ee440f7fe6f None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:46.218676 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef629348-e485-4572-a4b2-ec317f1d59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:46.219564 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629348-e485-4572-a4b2-ec317f1d59b6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:46.219846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629348-e485-4572-a4b2-ec317f1d59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:46.220122 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629348-e485-4572-a4b2-ec317f1d59b6 None admin] PATH_INFO: `/v3/projects/f9016a548ee34e14b91d58e5379543bc` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:46.221365 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef629348-e485-4572-a4b2-ec317f1d59b6 None admin] RBAC: Authorizing `identity:get_project(project_id=f9016a548ee34e14b91d58e5379543bc)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:46.222806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef629348-e485-4572-a4b2-ec317f1d59b6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:46.224505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 597/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:30:46 2022] GET /identity/v3/projects/f9016a548ee34e14b91d58e5379543bc => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:30:46.230769 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc8fdc-67a2-4b8c-911e-fcb56ec796da None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:30:46.330323 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc8fdc-67a2-4b8c-911e-fcb56ec796da None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:30:46.364364 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbd2c1-23a7-4beb-8c83-f5ed8b3de47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:30:46.365026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd2c1-23a7-4beb-8c83-f5ed8b3de47f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:30:46.365337 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd2c1-23a7-4beb-8c83-f5ed8b3de47f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:30:46.365618 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd2c1-23a7-4beb-8c83-f5ed8b3de47f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:30:46.366456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbd2c1-23a7-4beb-8c83-f5ed8b3de47f None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=f9016a548ee34e14b91d58e5379543bc, user.id=d6145fd5a11b493d934a55dafe044a3d) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:30:46.366772 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbd2c1-23a7-4beb-8c83-f5ed8b3de47f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:30:46.368210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbd2c1-23a7-4beb-8c83-f5ed8b3de47f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:30:46.374013 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 603/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:30:46 2022] GET /identity/v3/role_assignments?user.id=d6145fd5a11b493d934a55dafe044a3d&scope.project.id=f9016a548ee34e14b91d58e5379543bc&role.id=58c34600086c469b87bfa39b74dd2607 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:31:45.193879 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36b1f-f135-4fb8-9988-1053f9f8b1c4 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:45.194460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36b1f-f135-4fb8-9988-1053f9f8b1c4 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:45.194460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36b1f-f135-4fb8-9988-1053f9f8b1c4 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:45.195862 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 598/1201] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:31:45.201335 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22da78-aad7-4352-917c-9c6c389df850 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:45.201792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22da78-aad7-4352-917c-9c6c389df850 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:45.201885 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22da78-aad7-4352-917c-9c6c389df850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:45.219462 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-3a22da78-aad7-4352-917c-9c6c389df850 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:31:45.220720 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-3a22da78-aad7-4352-917c-9c6c389df850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:31:45.289217 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 604/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:31:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:31:45.314464 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb36b1f-f135-4fb8-9988-1053f9f8b1c4 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:45.435400 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb36b1f-f135-4fb8-9988-1053f9f8b1c4 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:45.475085 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:45.476021 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:45.476510 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:45.476881 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:45.490391 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:45.492002 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:31:45.510806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:45.511988 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:45.513078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:31:45.513078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:31:45.513078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:45.513078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:31:45.513078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 599/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:45.520985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22da78-aad7-4352-917c-9c6c389df850 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:45.631595 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22da78-aad7-4352-917c-9c6c389df850 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:45.673259 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fca4a-5c3b-47f6-9207-e34706202b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:45.673971 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fca4a-5c3b-47f6-9207-e34706202b4c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:45.674265 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fca4a-5c3b-47f6-9207-e34706202b4c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:45.674568 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fca4a-5c3b-47f6-9207-e34706202b4c None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:45.675309 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fca4a-5c3b-47f6-9207-e34706202b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:45.675620 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fca4a-5c3b-47f6-9207-e34706202b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:45.677007 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fca4a-5c3b-47f6-9207-e34706202b4c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:45.694359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 605/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:45.695957 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:45.831127 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794d05a-d4a4-4b41-80be-42d160e44eae None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:45.874518 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ff3e2-701d-46bc-a426-32569abbf73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:45.875477 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ff3e2-701d-46bc-a426-32569abbf73a None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:45.875821 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ff3e2-701d-46bc-a426-32569abbf73a None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:45.876195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ff3e2-701d-46bc-a426-32569abbf73a None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:45.877123 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ff3e2-701d-46bc-a426-32569abbf73a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:45.878953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ff3e2-701d-46bc-a426-32569abbf73a None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:45.909281 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 600/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:31:45 2022] POST /identity/v3/users => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:31:48.383346 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83101c06-67ae-49cd-a413-66f3cd380f2c None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:48.383932 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83101c06-67ae-49cd-a413-66f3cd380f2c None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:48.383932 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83101c06-67ae-49cd-a413-66f3cd380f2c None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:48.385314 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 606/1206] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:31:48.391256 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b431380-2937-410a-9a51-38911179ddb1 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:48.391704 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b431380-2937-410a-9a51-38911179ddb1 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:48.391809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b431380-2937-410a-9a51-38911179ddb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:48.412276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-2b431380-2937-410a-9a51-38911179ddb1 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:31:48.413596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-2b431380-2937-410a-9a51-38911179ddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:31:48.484713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 601/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:31:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:31:48.509605 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83101c06-67ae-49cd-a413-66f3cd380f2c None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:48.621158 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83101c06-67ae-49cd-a413-66f3cd380f2c None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:48.660396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034db06-ace9-45d2-8820-ef8c4b1ed244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:48.661409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034db06-ace9-45d2-8820-ef8c4b1ed244 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:48.661765 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034db06-ace9-45d2-8820-ef8c4b1ed244 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:48.662134 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034db06-ace9-45d2-8820-ef8c4b1ed244 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:48.676018 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034db06-ace9-45d2-8820-ef8c4b1ed244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-2034db06-ace9-45d2-8820-ef8c4b1ed244 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:31:48.683850 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:48.684926 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:48.686307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 607/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:31:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:31:48.692117 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b431380-2937-410a-9a51-38911179ddb1 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:48.812624 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b431380-2937-410a-9a51-38911179ddb1 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:48.852040 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:48.852622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:48.852932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:48.853222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:48.854001 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:48.854298 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:48.855938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:48.864888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 602/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:31:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:48.870581 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034db06-ace9-45d2-8820-ef8c4b1ed244 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:48.980916 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034db06-ace9-45d2-8820-ef8c4b1ed244 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:49.024832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:49.025782 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:49.026152 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:49.026451 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:49.037819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:49.039290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:31:49.057100 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:49.058203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:49.059169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:31:49.059169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:31:49.059169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:49.059169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:49.059169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 608/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:31:49.064452 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:49.186983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce757-e04a-4ef6-a318-45134438d368 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:49.227315 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:49.227971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:49.228254 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:49.228530 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:49.229295 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:49.229629 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:49.232634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:49.249898 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 603/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:49.251178 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:49.372729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc24a6-4ca4-4c8f-81b5-ae71267c1393 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:49.432742 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:49.433669 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:49.433864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:49.434261 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:49.443441 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:49.445179 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:31:49.460414 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:49.461812 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:49.463004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 609/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:49.468020 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:49.576152 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3aa8f-cb00-42f2-bae9-f2d0b7f037ea None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:49.616382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:49.617079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:49.617368 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:49.619863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:49.620746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:49.621087 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:49.622540 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:49.642561 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 604/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:31:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:49.651577 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:49.765886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292cdf5-c237-47f1-82ef-5739952b50ba None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:49.802792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec6674-3b04-402f-803c-32f0d267ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:49.803907 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec6674-3b04-402f-803c-32f0d267ce1c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:49.804200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec6674-3b04-402f-803c-32f0d267ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:49.804483 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec6674-3b04-402f-803c-32f0d267ce1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:49.879034 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec6674-3b04-402f-803c-32f0d267ce1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:49.880529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec6674-3b04-402f-803c-32f0d267ce1c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:49.986182 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 610/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:31:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:31:49.991718 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:50.097057 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36f5fa-6fce-4130-8410-88738bef7d02 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:50.136359 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:50.137248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:50.137541 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:50.137831 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:50.154379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:50.156080 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:31:50.177195 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:50.178663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:50.179907 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:31:50.179907 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:31:50.179907 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:50.179907 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:31:50.179907 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 605/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:50.184864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec6674-3b04-402f-803c-32f0d267ce1c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:50.299553 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec6674-3b04-402f-803c-32f0d267ce1c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:50.338256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:50.339004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:50.339322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:50.339599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:50.340341 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:50.340655 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:50.342246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:50.354750 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 611/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:31:50.360891 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:50.477523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a1eed-f0e0-474a-bd55-c6505ccfa245 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:50.524274 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:50.525089 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:50.525379 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:50.525653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:50.533500 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:50.535266 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:31:50.555294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:50.556842 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:31:50.557882 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 606/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:31:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:50.562180 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:50.689191 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0aa09-fd79-4e3d-a856-aadaf8f6c84e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:50.727395 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea077b0f-00db-490f-aeb1-b245fbf2b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:50.728288 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea077b0f-00db-490f-aeb1-b245fbf2b366 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:50.728449 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea077b0f-00db-490f-aeb1-b245fbf2b366 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:50.728686 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea077b0f-00db-490f-aeb1-b245fbf2b366 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:50.729585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea077b0f-00db-490f-aeb1-b245fbf2b366 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:50.729936 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea077b0f-00db-490f-aeb1-b245fbf2b366 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:50.731868 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea077b0f-00db-490f-aeb1-b245fbf2b366 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:50.746134 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 612/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:31:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:50.752858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:50.865839 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c5d67-41bd-4763-b765-c4493d64cbe7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:50.904763 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a7255-6576-4a21-a74d-1864db4b4967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:50.906215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a7255-6576-4a21-a74d-1864db4b4967 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:50.906702 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a7255-6576-4a21-a74d-1864db4b4967 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:50.907022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a7255-6576-4a21-a74d-1864db4b4967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:50.908071 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a7255-6576-4a21-a74d-1864db4b4967 None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=34415b5565ee42fea28ffb93825754b1) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:50.908522 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a7255-6576-4a21-a74d-1864db4b4967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:50.910271 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a7255-6576-4a21-a74d-1864db4b4967 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:50.916464 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 607/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:31:50 2022] GET /identity/v3/role_assignments?user.id=34415b5565ee42fea28ffb93825754b1&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:53.219985 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229e3f2-17c8-4ada-886e-792242fbfa09 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:53.220557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229e3f2-17c8-4ada-886e-792242fbfa09 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:53.220557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229e3f2-17c8-4ada-886e-792242fbfa09 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:53.222110 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 613/1220] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:31:53.229076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76789f02-2956-41a5-9cd2-2cfb55d5bc6f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:53.229076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76789f02-2956-41a5-9cd2-2cfb55d5bc6f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:53.229076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76789f02-2956-41a5-9cd2-2cfb55d5bc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:53.247256 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-76789f02-2956-41a5-9cd2-2cfb55d5bc6f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:31:53.248474 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-76789f02-2956-41a5-9cd2-2cfb55d5bc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:31:53.322604 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 608/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:31:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:31:53.347526 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229e3f2-17c8-4ada-886e-792242fbfa09 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:53.475425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229e3f2-17c8-4ada-886e-792242fbfa09 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:53.517640 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d07fe4-4a12-44a1-83ae-b7f79064866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:53.518695 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d07fe4-4a12-44a1-83ae-b7f79064866a None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:53.518983 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d07fe4-4a12-44a1-83ae-b7f79064866a None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:53.519258 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d07fe4-4a12-44a1-83ae-b7f79064866a None admin] PATH_INFO: `/v3/roles/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:53.533546 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d07fe4-4a12-44a1-83ae-b7f79064866a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-60d07fe4-4a12-44a1-83ae-b7f79064866a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:31:53.541954 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:53.543500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:53.544629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 614/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:31:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:53.550173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76789f02-2956-41a5-9cd2-2cfb55d5bc6f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:53.662039 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76789f02-2956-41a5-9cd2-2cfb55d5bc6f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:53.699007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:53.699682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:53.699961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:53.700231 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:53.700975 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] RBAC: Adding query filter params (name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:53.701323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:53.702729 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:53.710758 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 609/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:31:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:31:53.717838 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d07fe4-4a12-44a1-83ae-b7f79064866a None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:53.846492 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d07fe4-4a12-44a1-83ae-b7f79064866a None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:53.884839 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:53.885762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:53.886115 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:53.886393 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:53.896714 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:53.898199 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:31:53.916205 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:53.918010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:53.919855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:31:53.919855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:31:53.919855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:53.919855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:53.919855 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 615/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:53.924242 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:54.027081 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edb945-33b5-4e33-be73-047e3d0e8bfd None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:54.062888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:54.066885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:54.067291 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:54.067579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:54.068377 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:54.069513 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:54.070930 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:54.087911 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 610/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:54.094266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:54.196745 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139b08d-d48c-4e0e-8e1a-a94a3f2f88f3 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:54.231088 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:54.231971 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:54.232237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:54.232509 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:54.241237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:54.242707 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:31:54.257122 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:54.258392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:54.259456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 616/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:54.264936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:54.365108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514c6ac-7999-4b87-813b-5605108d2ee6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:54.402811 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:54.403508 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:54.403771 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:54.404047 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:54.404890 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:54.405226 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:54.406645 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:54.427634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 611/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:31:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:54.433408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:54.535968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f87e42-96a6-4bb6-bfdf-85a86b5121b2 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:54.569996 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:54.570794 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:54.571077 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:54.571351 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:54.582429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:54.583881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:31:54.601664 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:54.602896 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:54.604014 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:31:54.604014 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:31:54.604014 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:54.604014 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:54.604014 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 617/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:54.609719 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:54.714962 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f87eb-ef02-413e-b64e-ee916f464244 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:54.753620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:54.754705 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:54.755222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:54.755780 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:54.756769 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:54.757396 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:54.759330 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:54.773203 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 612/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:54.778832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:54.901674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a1a6-c4c4-4120-bdaa-ab21f507ad69 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:54.942755 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:54.943740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:54.944078 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:54.944408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:54.953687 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:54.955210 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:31:54.970214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:54.971585 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:54.972611 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 618/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:54.977599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:55.089650 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d6d3a-0d25-4e5e-b62b-d4364de8a162 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:55.127712 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da946b95-d0f4-43e6-b989-651c3de2d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:55.128385 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946b95-d0f4-43e6-b989-651c3de2d315 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:55.128673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946b95-d0f4-43e6-b989-651c3de2d315 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:55.128953 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946b95-d0f4-43e6-b989-651c3de2d315 None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:55.129804 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da946b95-d0f4-43e6-b989-651c3de2d315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:55.130120 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da946b95-d0f4-43e6-b989-651c3de2d315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:55.131578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da946b95-d0f4-43e6-b989-651c3de2d315 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:55.143789 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 613/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:55.150575 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:55.258116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e009c-82b3-441b-a90e-e534494e76e0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:55.292776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec29404-7bcd-4cb2-a519-9c913740d794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:55.293756 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec29404-7bcd-4cb2-a519-9c913740d794 None admin] REQUEST_METHOD: `PUT` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:55.294052 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec29404-7bcd-4cb2-a519-9c913740d794 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:55.294405 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec29404-7bcd-4cb2-a519-9c913740d794 None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/34415b5565ee42fea28ffb93825754b1/roles/582e6b7be6a445019c4cd90d2dcd594a` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:55.312105 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec29404-7bcd-4cb2-a519-9c913740d794 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=34415b5565ee42fea28ffb93825754b1, role_id=582e6b7be6a445019c4cd90d2dcd594a)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:55.313759 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec29404-7bcd-4cb2-a519-9c913740d794 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:55.326054 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 619/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:31:55 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/34415b5565ee42fea28ffb93825754b1/roles/582e6b7be6a445019c4cd90d2dcd594a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:31:57.585847 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db38f52-8840-4531-91f8-e48d79324487 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:57.586440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db38f52-8840-4531-91f8-e48d79324487 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:57.586440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db38f52-8840-4531-91f8-e48d79324487 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:57.587844 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 614/1233] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:31:57.593338 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49125307-dec8-4e56-b072-fe936b49cc8e None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:57.593796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49125307-dec8-4e56-b072-fe936b49cc8e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:57.593923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49125307-dec8-4e56-b072-fe936b49cc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:57.611301 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-49125307-dec8-4e56-b072-fe936b49cc8e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:31:57.612574 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-49125307-dec8-4e56-b072-fe936b49cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:31:57.678234 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 620/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:31:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:31:57.702885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db38f52-8840-4531-91f8-e48d79324487 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:57.832344 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db38f52-8840-4531-91f8-e48d79324487 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:57.867093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e321-07e1-461d-bd13-77d6d78c7968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:57.867950 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e321-07e1-461d-bd13-77d6d78c7968 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:57.868244 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e321-07e1-461d-bd13-77d6d78c7968 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:57.868652 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e321-07e1-461d-bd13-77d6d78c7968 None admin] PATH_INFO: `/v3/roles/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:57.881311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816e321-07e1-461d-bd13-77d6d78c7968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-8816e321-07e1-461d-bd13-77d6d78c7968 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:31:57.891239 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:57.899637 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:31:57.900863 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 615/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:31:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:57.902028 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49125307-dec8-4e56-b072-fe936b49cc8e None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:58.002817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49125307-dec8-4e56-b072-fe936b49cc8e None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:58.035967 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:58.036645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:58.036927 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:58.037238 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:58.038005 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] RBAC: Adding query filter params (name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:58.038313 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:58.039710 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:58.048762 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 621/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:31:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:31:58.055174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e321-07e1-461d-bd13-77d6d78c7968 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:58.178928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e321-07e1-461d-bd13-77d6d78c7968 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:58.218673 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:58.219546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:58.219835 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:58.220106 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:58.231068 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:58.232534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:31:58.250436 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:58.251597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:58.252858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:31:58.252858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:31:58.252858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:58.252858 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:31:58.256468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 616/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:58.257815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:58.370724 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59b5f0-1fd3-407b-8661-bce4ddb28723 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:58.407858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:58.408598 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:58.408881 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:58.409292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:58.410058 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:58.410402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:58.412059 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:58.426056 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 622/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:58.432102 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:58.536680 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bde54-febb-4940-a60f-696540bbcbd6 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:58.570596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:58.571571 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:58.571866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:58.572157 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:58.581377 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:58.583015 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:31:58.600004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:58.606921 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:31:58.608877 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 617/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:58.609976 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:58.712902 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64db27-5c5c-4e44-b75e-67b787db9e92 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:58.751412 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:58.752186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:58.752481 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:58.752860 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:58.753843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:58.754271 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:58.755777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:58.776005 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 623/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:31:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:58.782866 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:58.888697 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20226b-6388-43c1-bdc2-c24a7a3219b8 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:58.927685 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1f93-618c-4a2a-9709-d419be7c5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:58.928406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1f93-618c-4a2a-9709-d419be7c5430 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:58.928694 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1f93-618c-4a2a-9709-d419be7c5430 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:58.929086 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1f93-618c-4a2a-9709-d419be7c5430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:59.003612 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1f93-618c-4a2a-9709-d419be7c5430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:59.005089 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1f93-618c-4a2a-9709-d419be7c5430 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:59.111600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 618/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:31:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:31:59.117722 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:59.224005 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1675d-ed43-481e-ad23-3ba4a28cf3a1 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:59.263315 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:59.264201 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:59.265203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:59.265618 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:59.277444 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:59.279246 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:31:59.297841 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:59.299113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:59.300308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:31:59.300308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:31:59.300308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:31:59.300308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:59.300308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 624/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:31:59.305665 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1f93-618c-4a2a-9709-d419be7c5430 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:59.412711 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1f93-618c-4a2a-9709-d419be7c5430 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:59.448984 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:59.449751 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:59.450105 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:59.450468 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:59.451204 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:59.451519 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:59.453862 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:59.466222 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 619/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:31:59.472149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:59.578237 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f237cd3-dc68-47af-8915-bbf17f154308 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:59.613687 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:59.614567 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:59.614962 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:59.615165 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:59.622308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:59.624363 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:31:59.638084 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:31:59.639308 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:31:59.640583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 625/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:31:59.648167 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:59.761349 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cad53-eddb-4402-983e-f3c3457c3512 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:59.803110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710affe-34d1-48ba-8220-5977efa23bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:59.803859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710affe-34d1-48ba-8220-5977efa23bab None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:59.804215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710affe-34d1-48ba-8220-5977efa23bab None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:59.804589 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710affe-34d1-48ba-8220-5977efa23bab None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:59.805475 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710affe-34d1-48ba-8220-5977efa23bab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:59.805793 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710affe-34d1-48ba-8220-5977efa23bab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:59.807463 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710affe-34d1-48ba-8220-5977efa23bab None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:59.821494 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 620/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:31:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:31:59.827890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:31:59.933521 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7bcb41-39a3-4ec7-9db4-0f4e3fa30bd2 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:31:59.970875 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf363143-e22d-4b5b-bb22-97f07376c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:31:59.971543 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363143-e22d-4b5b-bb22-97f07376c00f None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:31:59.971827 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363143-e22d-4b5b-bb22-97f07376c00f None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:31:59.972098 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363143-e22d-4b5b-bb22-97f07376c00f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:31:59.972978 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf363143-e22d-4b5b-bb22-97f07376c00f None admin] RBAC: Adding query filter params (role.id=582e6b7be6a445019c4cd90d2dcd594a, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=34415b5565ee42fea28ffb93825754b1) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:31:59.973322 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf363143-e22d-4b5b-bb22-97f07376c00f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:31:59.974777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf363143-e22d-4b5b-bb22-97f07376c00f None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:31:59.980793 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 626/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:31:59 2022] GET /identity/v3/role_assignments?user.id=34415b5565ee42fea28ffb93825754b1&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=582e6b7be6a445019c4cd90d2dcd594a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:02.376676 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93321d9b-b5a9-470f-983b-10df57811fe5 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:02.377237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93321d9b-b5a9-470f-983b-10df57811fe5 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:02.377237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93321d9b-b5a9-470f-983b-10df57811fe5 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:02.378847 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 621/1247] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:32:02.384335 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd53ef-cf13-4377-9a8c-bb347ce8d14d None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:02.384805 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd53ef-cf13-4377-9a8c-bb347ce8d14d None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:02.384912 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd53ef-cf13-4377-9a8c-bb347ce8d14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:02.402304 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-a2fd53ef-cf13-4377-9a8c-bb347ce8d14d None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:32:02.403486 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-a2fd53ef-cf13-4377-9a8c-bb347ce8d14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:32:02.474137 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 627/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:32:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:32:02.503461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93321d9b-b5a9-470f-983b-10df57811fe5 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:02.617174 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93321d9b-b5a9-470f-983b-10df57811fe5 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:02.655976 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d10011-4030-47d5-af2e-eeeb4e52fb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:02.656771 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d10011-4030-47d5-af2e-eeeb4e52fb4e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:02.657084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d10011-4030-47d5-af2e-eeeb4e52fb4e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:02.657357 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d10011-4030-47d5-af2e-eeeb4e52fb4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:02.667596 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d10011-4030-47d5-af2e-eeeb4e52fb4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-f9d10011-4030-47d5-af2e-eeeb4e52fb4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:32:02.675915 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:02.677405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:02.678460 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 622/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:32:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:02.683597 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd53ef-cf13-4377-9a8c-bb347ce8d14d None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:02.797010 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd53ef-cf13-4377-9a8c-bb347ce8d14d None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:02.833683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:02.834422 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:02.834738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:02.834979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:02.835789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:02.836097 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:02.837550 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:02.847144 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 628/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:32:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:02.853521 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d10011-4030-47d5-af2e-eeeb4e52fb4e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:02.967997 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d10011-4030-47d5-af2e-eeeb4e52fb4e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:03.005323 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:03.006125 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:03.006409 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:03.006685 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:03.018432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:03.019982 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:32:03.038278 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:03.039504 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:03.040692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:32:03.040692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:32:03.040692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:03.040692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:03.040692 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 623/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:03.047139 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:03.150004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c190c-898d-4ab9-be02-de2a0b8da675 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:03.185615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:03.186356 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:03.186647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:03.186929 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:03.187685 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:03.187979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:03.189380 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:03.202522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 629/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:03.208094 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:03.320012 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c6708-9137-4d9d-b8c3-e453fd1ddcc7 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:03.354010 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:03.354809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:03.355092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:03.355368 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:03.362928 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:03.364380 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:32:03.378263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:03.379971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:32:03.381191 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:03.381932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 624/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:03.386709 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:03.497696 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce8bc0-2a7e-459e-bdcf-b992256233cb None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:03.537609 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:03.538425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:03.538738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:03.539150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:03.540029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:03.540402 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:03.542187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:03.560803 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 630/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:32:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:03.567760 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:03.677996 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a993b5-3971-4dbf-867b-354cdcac2c07 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:03.718038 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662dc7a-1836-4599-a552-06bd08ae2382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:03.718925 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662dc7a-1836-4599-a552-06bd08ae2382 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:03.719340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662dc7a-1836-4599-a552-06bd08ae2382 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:03.719728 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662dc7a-1836-4599-a552-06bd08ae2382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:03.809004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662dc7a-1836-4599-a552-06bd08ae2382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:03.810961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662dc7a-1836-4599-a552-06bd08ae2382 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:03.917283 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 625/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:32:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:32:03.923501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:04.025026 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebae99d-6f1c-4101-9654-257de84a9768 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:04.061679 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:04.062730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:04.063011 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:04.063292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:04.073256 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:04.074711 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:32:04.091957 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:04.093181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:04.094243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:32:04.094243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:32:04.094243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:04.094243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:04.094243 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 631/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:32:04.099672 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662dc7a-1836-4599-a552-06bd08ae2382 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:04.204740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662dc7a-1836-4599-a552-06bd08ae2382 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:04.241408 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:04.242076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:04.242362 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:04.242689 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:04.243520 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:04.243859 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:04.245419 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:04.258357 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 626/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:04.264137 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:04.372434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d50013-6422-49de-a433-2e7c6dfa1ce8 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:04.408214 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:04.409069 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:04.409353 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:04.409618 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:04.417216 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:04.418701 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:32:04.434407 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:04.435713 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:04.436865 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 632/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:32:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:04.441844 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:04.547844 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f349bb9-5f94-41cb-a5ce-f32e52094fa3 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:04.581579 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4afa800-4e6f-4913-bef7-ea1f6a7dd7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:04.582251 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afa800-4e6f-4913-bef7-ea1f6a7dd7cd None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:04.582533 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afa800-4e6f-4913-bef7-ea1f6a7dd7cd None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:04.582810 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4afa800-4e6f-4913-bef7-ea1f6a7dd7cd None admin] PATH_INFO: `/v3/projects` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:04.583620 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afa800-4e6f-4913-bef7-ea1f6a7dd7cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:04.583934 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afa800-4e6f-4913-bef7-ea1f6a7dd7cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:04.585431 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4afa800-4e6f-4913-bef7-ea1f6a7dd7cd None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:04.598919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 627/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:04.605228 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:04.706649 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5a054-ce6e-45a3-ac4b-6b0e0813ed74 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:04.748645 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d44fa-64b3-4209-92e9-4d581cec3348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:04.749793 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d44fa-64b3-4209-92e9-4d581cec3348 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:04.751125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d44fa-64b3-4209-92e9-4d581cec3348 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:04.751125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d44fa-64b3-4209-92e9-4d581cec3348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:04.751614 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d44fa-64b3-4209-92e9-4d581cec3348 None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=34415b5565ee42fea28ffb93825754b1) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:04.751988 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d44fa-64b3-4209-92e9-4d581cec3348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:04.753873 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d44fa-64b3-4209-92e9-4d581cec3348 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:04.762295 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 633/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:32:04 2022] GET /identity/v3/role_assignments?user.id=34415b5565ee42fea28ffb93825754b1&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=58c34600086c469b87bfa39b74dd2607 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:32:06.968108 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffaa51-207d-4c14-bd5a-072402b70791 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:06.968671 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffaa51-207d-4c14-bd5a-072402b70791 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:06.968671 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffaa51-207d-4c14-bd5a-072402b70791 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:06.970076 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 628/1261] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:32:06.975887 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841b06-08ff-42c6-b4c6-ccfb5dcc4e93 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:06.976268 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841b06-08ff-42c6-b4c6-ccfb5dcc4e93 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:06.976440 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66841b06-08ff-42c6-b4c6-ccfb5dcc4e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:06.993267 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-66841b06-08ff-42c6-b4c6-ccfb5dcc4e93 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:32:06.994448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-66841b06-08ff-42c6-b4c6-ccfb5dcc4e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:32:07.058083 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 634/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:32:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:32:07.081472 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffaa51-207d-4c14-bd5a-072402b70791 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:07.196797 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffaa51-207d-4c14-bd5a-072402b70791 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:07.229215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc096a2-b3d0-45e8-adab-90633eaca26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:07.229936 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc096a2-b3d0-45e8-adab-90633eaca26e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:07.230248 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc096a2-b3d0-45e8-adab-90633eaca26e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:07.230780 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc096a2-b3d0-45e8-adab-90633eaca26e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:07.240397 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc096a2-b3d0-45e8-adab-90633eaca26e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-3bc096a2-b3d0-45e8-adab-90633eaca26e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:32:07.247888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:07.249177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:07.250534 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 629/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:32:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:07.255171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66841b06-08ff-42c6-b4c6-ccfb5dcc4e93 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:07.348043 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66841b06-08ff-42c6-b4c6-ccfb5dcc4e93 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:07.380946 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:07.381627 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:07.381907 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:07.382177 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] PATH_INFO: `/v3/roles` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:07.382916 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:07.383219 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:07.384588 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:07.392754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 635/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:32:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:07.398581 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc096a2-b3d0-45e8-adab-90633eaca26e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:07.510190 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc096a2-b3d0-45e8-adab-90633eaca26e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:07.541559 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:07.542335 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:07.542616 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:07.542886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:07.552870 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:07.554342 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:32:07.571022 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:07.572134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:07.573542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:32:07.573542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:32:07.573542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:07.573542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:07.573542 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 630/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:07.578482 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:07.672126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2fd02-f3d5-4468-863a-82715b47a659 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:07.704320 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:07.704966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:07.705277 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:07.705553 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:07.706268 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:07.706573 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:07.707932 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:07.719522 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 636/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:07.726171 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:07.826802 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384957c4-dbfc-4128-bae0-28fc3111b7e8 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:07.858634 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:07.859426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:07.859713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:07.859995 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] PATH_INFO: `/v3/users/placement` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:07.867249 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:07.868746 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:32:07.883405 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:07.884611 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:07.885938 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 631/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:07.890633 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:07.986480 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bbcb4-6031-4f7a-9bd7-05062e6f5a7c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:08.018731 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:08.019394 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:08.019681 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:08.019960 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] PATH_INFO: `/v3/users` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:08.020801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:08.021145 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:08.022584 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:08.038824 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 637/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:32:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:08.044353 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:08.143234 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d2810-7d91-4873-aafb-f64069a02c43 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:08.174142 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:08.174919 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:08.175201 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:08.175476 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:08.185459 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:08.187401 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:32:08.204252 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:08.205411 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:08.206618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:32:08.206618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:32:08.206618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:08.206618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:08.206618 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 632/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:08.211086 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:08.304702 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e7a1-92f2-443a-9e77-53f2c977de17 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:08.337299 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:08.337948 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:08.338235 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:08.338512 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:08.339241 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:08.339552 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:08.340920 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:08.351711 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 638/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:08.357292 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:08.452500 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a6db6-a61d-4406-b1e3-bdda11c78046 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:08.484030 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:08.484797 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:08.485110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:08.485384 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:08.491713 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:08.493173 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:32:08.507233 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:08.508424 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:08.509450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 633/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:32:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:08.514117 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:08.608962 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c59d8-41ce-4dfc-ade8-69109a033f69 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:08.640714 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd606f7-5b4a-4dfb-b90b-84fa4c4a8e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:08.641398 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd606f7-5b4a-4dfb-b90b-84fa4c4a8e3c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:08.641690 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd606f7-5b4a-4dfb-b90b-84fa4c4a8e3c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:08.641968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd606f7-5b4a-4dfb-b90b-84fa4c4a8e3c None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:08.642760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd606f7-5b4a-4dfb-b90b-84fa4c4a8e3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:08.643073 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd606f7-5b4a-4dfb-b90b-84fa4c4a8e3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:08.644534 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd606f7-5b4a-4dfb-b90b-84fa4c4a8e3c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:08.655947 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 639/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:32:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:08.661635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:08.763819 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2f760-39b5-42b1-904a-34b910711f05 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:08.795775 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1a350-bbfa-4245-9d32-fecfc343a0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:08.796647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a350-bbfa-4245-9d32-fecfc343a0aa None admin] REQUEST_METHOD: `PUT` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:08.796926 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a350-bbfa-4245-9d32-fecfc343a0aa None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:08.797227 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1a350-bbfa-4245-9d32-fecfc343a0aa None admin] PATH_INFO: `/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/34415b5565ee42fea28ffb93825754b1/roles/58c34600086c469b87bfa39b74dd2607` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:08.799158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1a350-bbfa-4245-9d32-fecfc343a0aa None admin] RBAC: Authorizing `identity:create_grant(project_id=b5d4f68f7dc4464a901b1d2d6b617c68, user_id=34415b5565ee42fea28ffb93825754b1, role_id=58c34600086c469b87bfa39b74dd2607)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:08.800570 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1a350-bbfa-4245-9d32-fecfc343a0aa None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:08.813384 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 634/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:32:08 2022] PUT /identity/v3/projects/b5d4f68f7dc4464a901b1d2d6b617c68/users/34415b5565ee42fea28ffb93825754b1/roles/58c34600086c469b87bfa39b74dd2607 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:32:11.029349 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd8148-f1bc-4eff-b4b9-79057ce7c33f None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:11.029919 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd8148-f1bc-4eff-b4b9-79057ce7c33f None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:11.029919 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd8148-f1bc-4eff-b4b9-79057ce7c33f None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:11.031298 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 640/1274] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:32:11.036606 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87734a-d61f-4740-921b-295603ea7b47 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:11.037072 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87734a-d61f-4740-921b-295603ea7b47 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:11.037205 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e87734a-d61f-4740-921b-295603ea7b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:11.054119 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-1e87734a-d61f-4740-921b-295603ea7b47 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:32:11.055322 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-1e87734a-d61f-4740-921b-295603ea7b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:32:11.120211 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 635/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:32:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:32:11.143727 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd8148-f1bc-4eff-b4b9-79057ce7c33f None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:11.241505 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd8148-f1bc-4eff-b4b9-79057ce7c33f None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:11.273392 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41101d-109a-42e2-af59-9d1a59a426bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:11.274171 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41101d-109a-42e2-af59-9d1a59a426bf None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:11.274458 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41101d-109a-42e2-af59-9d1a59a426bf None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:11.274734 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41101d-109a-42e2-af59-9d1a59a426bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:11.284965 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41101d-109a-42e2-af59-9d1a59a426bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-cd41101d-109a-42e2-af59-9d1a59a426bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:32:11.292508 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:11.293716 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:11.294826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 641/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:32:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:11.300099 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87734a-d61f-4740-921b-295603ea7b47 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:11.393576 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e87734a-d61f-4740-921b-295603ea7b47 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:11.426506 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:11.427169 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:11.427455 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:11.427731 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] PATH_INFO: `/v3/roles` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:11.428478 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:11.428791 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:11.430190 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:11.437966 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 636/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:32:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:11.443777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41101d-109a-42e2-af59-9d1a59a426bf None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:11.538980 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41101d-109a-42e2-af59-9d1a59a426bf None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:11.569912 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:11.570683 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:11.570966 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:11.571240 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:11.581819 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:11.583260 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:32:11.600082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:11.601307 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:11.602529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:32:11.602529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:32:11.602529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:11.602529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:11.602529 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 642/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:11.607144 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:11.699892 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d5333-89ab-41b8-a3aa-75deb08d5f48 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:11.738110 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:11.739021 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:11.739304 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:11.739772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] PATH_INFO: `/v3/domains` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:11.740708 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:11.741158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:11.743053 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:11.755293 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 637/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:11.761115 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:11.855801 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9c03a-1415-4dd4-983a-00e2c726092e None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:11.887162 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:11.887942 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:11.888225 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:11.888500 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:11.896189 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:11.897662 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:32:11.911465 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:11.912654 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:11.913951 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 643/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:11.918366 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:12.015211 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ec9d2-d196-4ae1-b1bb-91dfe38ab757 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:12.047154 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:12.047809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:12.048091 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:12.048365 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] PATH_INFO: `/v3/users` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:12.049224 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:12.049533 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:12.050961 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:12.067448 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 638/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:32:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:12.073290 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:12.172488 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bb5c6-a41b-4ed0-8105-b471156c45aa None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:12.204652 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40a3bc-acb1-4108-be6d-92d261e574f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:12.205316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a3bc-acb1-4108-be6d-92d261e574f0 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:12.205600 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a3bc-acb1-4108-be6d-92d261e574f0 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:12.205874 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40a3bc-acb1-4108-be6d-92d261e574f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:12.269249 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40a3bc-acb1-4108-be6d-92d261e574f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:12.270711 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40a3bc-acb1-4108-be6d-92d261e574f0 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:12.363175 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 644/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:32:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:32:12.368886 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:12.467597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949782b3-d690-4331-be72-6cc235c1b1e9 None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:12.500433 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:12.501232 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:12.501543 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:12.501809 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:12.511270 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:12.512700 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:32:12.529263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:12.530406 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:12.531706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:32:12.531706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:32:12.531706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:32:12.531706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:12.531706 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 639/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:12.536184 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40a3bc-acb1-4108-be6d-92d261e574f0 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:12.629450 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40a3bc-acb1-4108-be6d-92d261e574f0 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:12.660930 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:12.661603 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:12.661886 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:12.662161 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] PATH_INFO: `/v3/domains` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:12.662876 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:12.663181 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:12.664558 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:12.675760 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 645/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:12.681443 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:12.778856 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebc786-36b0-420d-99ed-bbaf461af12d None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:12.811480 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:12.812254 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:12.812535 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:12.812807 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:12.819163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:12.820590 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:32:12.833778 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application with Lock( May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:12.834914 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: ERROR keystone.server.flask.application May 09 15:32:12.836116 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 640/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:32:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:12.840904 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:12.934880 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed972e0-821f-4428-80c1-fa4a1c4535b5 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:12.966232 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb85687-173f-47ad-bcb5-df7658fd8f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:12.966878 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb85687-173f-47ad-bcb5-df7658fd8f40 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:12.967728 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb85687-173f-47ad-bcb5-df7658fd8f40 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:12.968004 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb85687-173f-47ad-bcb5-df7658fd8f40 None admin] PATH_INFO: `/v3/projects` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:12.968790 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb85687-173f-47ad-bcb5-df7658fd8f40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:12.969135 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb85687-173f-47ad-bcb5-df7658fd8f40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:12.970553 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb85687-173f-47ad-bcb5-df7658fd8f40 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:12.982116 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 646/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:32:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:12.987885 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:13.082733 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1cc0e-dbf7-4d7d-a9d4-6e2e01a07e5c None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:13.116263 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda2226-8a5d-4207-9c4d-0533ad9ad4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:13.116918 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda2226-8a5d-4207-9c4d-0533ad9ad4de None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:13.117230 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda2226-8a5d-4207-9c4d-0533ad9ad4de None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:13.117505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda2226-8a5d-4207-9c4d-0533ad9ad4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:13.118337 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda2226-8a5d-4207-9c4d-0533ad9ad4de None admin] RBAC: Adding query filter params (role.id=58c34600086c469b87bfa39b74dd2607, scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68, user.id=34415b5565ee42fea28ffb93825754b1) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:13.118650 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda2226-8a5d-4207-9c4d-0533ad9ad4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:13.120067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda2226-8a5d-4207-9c4d-0533ad9ad4de None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:13.125772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 641/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:32:12 2022] GET /identity/v3/role_assignments?user.id=34415b5565ee42fea28ffb93825754b1&scope.project.id=b5d4f68f7dc4464a901b1d2d6b617c68&role.id=58c34600086c469b87bfa39b74dd2607 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:15.347628 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce141-00ad-45b3-8df1-0e2663bc362d None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:15.348200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce141-00ad-45b3-8df1-0e2663bc362d None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:15.348200 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdce141-00ad-45b3-8df1-0e2663bc362d None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:15.349615 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 647/1288] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:32:15.354715 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb504b-0fbe-44de-a557-72b7eb1ccd5f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:15.355118 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb504b-0fbe-44de-a557-72b7eb1ccd5f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:15.355277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb504b-0fbe-44de-a557-72b7eb1ccd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:15.372355 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-7aeb504b-0fbe-44de-a557-72b7eb1ccd5f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:32:15.374189 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-7aeb504b-0fbe-44de-a557-72b7eb1ccd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:32:15.447832 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 642/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:32:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:32:15.469532 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdce141-00ad-45b3-8df1-0e2663bc362d None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:15.565729 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdce141-00ad-45b3-8df1-0e2663bc362d None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:15.597408 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:15.598191 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:15.598524 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:15.598747 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] PATH_INFO: `/v3/services/placement` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:15.599396 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:15.600754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:15.613029 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:15.614359 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:15.615501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 648/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:32:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:15.620277 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb504b-0fbe-44de-a557-72b7eb1ccd5f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:15.711628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb504b-0fbe-44de-a557-72b7eb1ccd5f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:15.755852 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a311f1-9934-40d7-b75f-667dd13262d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:15.756940 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a311f1-9934-40d7-b75f-667dd13262d9 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:15.759036 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a311f1-9934-40d7-b75f-667dd13262d9 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:15.759350 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a311f1-9934-40d7-b75f-667dd13262d9 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:15.760084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a311f1-9934-40d7-b75f-667dd13262d9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:15.760395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a311f1-9934-40d7-b75f-667dd13262d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:15.761983 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a311f1-9934-40d7-b75f-667dd13262d9 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:15.771897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 643/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:32:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:15.778242 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:15.876085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd3657-979b-448e-9a65-377c2fb2b68c None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:15.907357 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a6277-e32d-4a88-9dbb-06382b35c0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:15.908007 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6277-e32d-4a88-9dbb-06382b35c0a6 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:15.908294 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6277-e32d-4a88-9dbb-06382b35c0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:15.908568 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a6277-e32d-4a88-9dbb-06382b35c0a6 None admin] PATH_INFO: `/v3/services` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:15.909299 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a6277-e32d-4a88-9dbb-06382b35c0a6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:15.909607 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a6277-e32d-4a88-9dbb-06382b35c0a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:15.910970 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a6277-e32d-4a88-9dbb-06382b35c0a6 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:15.915169 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 649/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:32:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:18.122835 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c61fb1-7892-4f03-ba4c-4ec5895b6201 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:18.123806 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c61fb1-7892-4f03-ba4c-4ec5895b6201 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:18.123979 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c61fb1-7892-4f03-ba4c-4ec5895b6201 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:18.125388 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 644/1293] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:32:18.130637 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8240-b820-4fbc-a326-32b12c5fd650 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:18.131064 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8240-b820-4fbc-a326-32b12c5fd650 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:18.131194 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8240-b820-4fbc-a326-32b12c5fd650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:18.148203 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-febc8240-b820-4fbc-a326-32b12c5fd650 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:32:18.149424 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-febc8240-b820-4fbc-a326-32b12c5fd650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:32:18.212398 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 650/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:32:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:32:18.234776 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c61fb1-7892-4f03-ba4c-4ec5895b6201 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:18.331318 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c61fb1-7892-4f03-ba4c-4ec5895b6201 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:18.363772 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29c444-c2c8-4fb4-918a-35b39cf74306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:18.364366 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29c444-c2c8-4fb4-918a-35b39cf74306 None admin] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:18.364654 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29c444-c2c8-4fb4-918a-35b39cf74306 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:18.364932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29c444-c2c8-4fb4-918a-35b39cf74306 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:18.365699 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29c444-c2c8-4fb4-918a-35b39cf74306 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:18.367072 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29c444-c2c8-4fb4-918a-35b39cf74306 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:18.383215 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 645/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:32:18 2022] POST /identity/v3/services => generated 246 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:32:20.598149 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b824a-29ee-4c91-a73c-1b820834e9e0 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:20.598688 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b824a-29ee-4c91-a73c-1b820834e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:20.598816 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b824a-29ee-4c91-a73c-1b820834e9e0 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:20.600102 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 651/1296] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:32:20.605320 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fec027-b8e5-41af-b5ea-2cedd6eb273f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:20.605958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fec027-b8e5-41af-b5ea-2cedd6eb273f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:20.606150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fec027-b8e5-41af-b5ea-2cedd6eb273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:20.622857 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-82fec027-b8e5-41af-b5ea-2cedd6eb273f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:32:20.624587 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-82fec027-b8e5-41af-b5ea-2cedd6eb273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:32:20.692690 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 646/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:32:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:32:20.714514 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b824a-29ee-4c91-a73c-1b820834e9e0 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:20.822834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b824a-29ee-4c91-a73c-1b820834e9e0 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:20.854425 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:20.855155 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:20.855691 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:20.855959 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] PATH_INFO: `/v3/services/placement` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:20.856609 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:20.858582 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:20.869945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:20.871187 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:20.872374 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 652/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:32:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:32:20.877285 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fec027-b8e5-41af-b5ea-2cedd6eb273f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:20.970311 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fec027-b8e5-41af-b5ea-2cedd6eb273f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:21.002353 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:21.003008 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:21.003294 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:21.003568 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:21.004275 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] RBAC: Adding query filter params (name=placement) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:21.004586 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:21.005981 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:21.010825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 647/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:32:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:21.016590 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:21.112442 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea785855-9e3d-4f66-8d8e-2c5e9a6e0393 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:21.143914 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8f426-7d32-4e53-9bef-95b6b70bdfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:21.144557 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8f426-7d32-4e53-9bef-95b6b70bdfdd None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:21.144836 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8f426-7d32-4e53-9bef-95b6b70bdfdd None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:21.145141 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8f426-7d32-4e53-9bef-95b6b70bdfdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:21.145923 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8f426-7d32-4e53-9bef-95b6b70bdfdd None admin] RBAC: Adding query filter params (interface=public, service_id=e7346af3aa994dbb8591664b2371722d, region_id=RegionOne) {{(pid=206280) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:21.146233 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8f426-7d32-4e53-9bef-95b6b70bdfdd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:21.147591 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8f426-7d32-4e53-9bef-95b6b70bdfdd None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:21.152428 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 653/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:32:21 2022] GET /identity/v3/endpoints?service_id=e7346af3aa994dbb8591664b2371722d&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:32:21.158505 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:21.252109 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d016f10-3ba8-45ea-b7c8-6c564636a33e None admin] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:21.284485 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584b86c-e982-4aac-8e1c-523a2112ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:21.285155 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584b86c-e982-4aac-8e1c-523a2112ff40 None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:21.285444 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584b86c-e982-4aac-8e1c-523a2112ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:21.285721 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584b86c-e982-4aac-8e1c-523a2112ff40 None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:21.286403 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584b86c-e982-4aac-8e1c-523a2112ff40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:21.287758 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584b86c-e982-4aac-8e1c-523a2112ff40 None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:21.294185 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 648/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:32:21 2022] GET /identity/v3/services => generated 2191 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:32:23.512085 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16271635-15fe-433e-b323-153bddd79905 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:23.512638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16271635-15fe-433e-b323-153bddd79905 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:23.512638 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16271635-15fe-433e-b323-153bddd79905 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:23.514060 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 654/1302] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:32:23.519286 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4a24a-1c93-4490-a236-c49bf1af4499 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:23.519707 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4a24a-1c93-4490-a236-c49bf1af4499 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:23.519820 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4a24a-1c93-4490-a236-c49bf1af4499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:23.536134 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-a3a4a24a-1c93-4490-a236-c49bf1af4499 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:32:23.537333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-a3a4a24a-1c93-4490-a236-c49bf1af4499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:32:23.601256 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 649/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:32:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:32:23.624268 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16271635-15fe-433e-b323-153bddd79905 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:23.721305 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16271635-15fe-433e-b323-153bddd79905 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:23.753884 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:23.754667 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:23.754947 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:23.755218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] PATH_INFO: `/v3/services/placement` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:23.755876 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:23.757263 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:32:23.768788 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:32:23.770123 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application with Lock( May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self._enter() May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application return self.creator() May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application created_value = creator( May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: ERROR keystone.server.flask.application May 09 15:32:23.771377 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 655/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:32:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:32:23.775966 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4a24a-1c93-4490-a236-c49bf1af4499 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:23.868783 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4a24a-1c93-4490-a236-c49bf1af4499 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:23.900416 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ad67b-d9d0-48b3-bc56-5e101b272bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:23.901084 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ad67b-d9d0-48b3-bc56-5e101b272bbf None admin] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:23.901371 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ad67b-d9d0-48b3-bc56-5e101b272bbf None admin] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:23.901644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ad67b-d9d0-48b3-bc56-5e101b272bbf None admin] PATH_INFO: `/v3/services` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:23.902340 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ad67b-d9d0-48b3-bc56-5e101b272bbf None admin] RBAC: Adding query filter params (name=placement) {{(pid=206281) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:32:23.902647 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ad67b-d9d0-48b3-bc56-5e101b272bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:23.904007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ad67b-d9d0-48b3-bc56-5e101b272bbf None admin] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:23.909358 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 650/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:32:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:32:23.914903 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:32:24.011244 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2eb2b-43bd-4faa-ada1-c1560944b358 None admin] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:32:24.042950 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af209b5c-3726-47e3-ae50-eb06f78e5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c2f4bd321b4a50a016143bb5412e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:32:24.043525 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af209b5c-3726-47e3-ae50-eb06f78e5081 None admin] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:32:24.043804 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af209b5c-3726-47e3-ae50-eb06f78e5081 None admin] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:32:24.044078 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af209b5c-3726-47e3-ae50-eb06f78e5081 None admin] PATH_INFO: `/v3/endpoints` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:32:24.045599 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af209b5c-3726-47e3-ae50-eb06f78e5081 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:32:24.046763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af209b5c-3726-47e3-ae50-eb06f78e5081 None admin] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:32:24.064055 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 656/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:32:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:33:29.478078 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385f82e-f4ad-4452-99c1-b5208850e825 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:29.478643 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385f82e-f4ad-4452-99c1-b5208850e825 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:29.478739 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385f82e-f4ad-4452-99c1-b5208850e825 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:29.480594 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 651/1307] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:29.486125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e7a2-9f84-4222-a4ac-82a583b0bddc None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:29.487777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e7a2-9f84-4222-a4ac-82a583b0bddc None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:29.487777 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e7a2-9f84-4222-a4ac-82a583b0bddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:29.644817 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-3f57e7a2-9f84-4222-a4ac-82a583b0bddc None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:29.646320 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-3f57e7a2-9f84-4222-a4ac-82a583b0bddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:29.849832 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 657/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:29.926685 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086abdf-c146-4449-9691-930a664fc1ed None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:29.926973 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086abdf-c146-4449-9691-930a664fc1ed None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:29.927254 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086abdf-c146-4449-9691-930a664fc1ed None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:29.928601 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 652/1309] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 9 15:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:29.936604 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7f88-190c-497c-92d5-8707184cda8d None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:29.936899 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7f88-190c-497c-92d5-8707184cda8d None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:29.937211 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7f88-190c-497c-92d5-8707184cda8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:30.102933 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-f7ae7f88-190c-497c-92d5-8707184cda8d None None] MFA Rules not processed for user `5382d7219c244430ae6f37d6b32e7d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:30.104270 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-f7ae7f88-190c-497c-92d5-8707184cda8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:30.302220 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086abdf-c146-4449-9691-930a664fc1ed None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:30.303769 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 658/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 9 15:33:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:33:30.304201 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-a086abdf-c146-4449-9691-930a664fc1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:30.518267 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086abdf-c146-4449-9691-930a664fc1ed None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:30.680309 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e189b0-8048-433e-be74-1a085b09ef29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:30.680978 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e189b0-8048-433e-be74-1a085b09ef29 service neutron] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:30.681304 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e189b0-8048-433e-be74-1a085b09ef29 service neutron] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:30.681552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e189b0-8048-433e-be74-1a085b09ef29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:30.694074 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-f9e189b0-8048-433e-be74-1a085b09ef29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:30.938025 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e189b0-8048-433e-be74-1a085b09ef29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:30.939648 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e189b0-8048-433e-be74-1a085b09ef29 service neutron] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:30.991657 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 653/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 15:33:33.793536 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa900fef-a03d-4536-ac99-4584882994d8 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:33.794706 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa900fef-a03d-4536-ac99-4584882994d8 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:33.795242 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa900fef-a03d-4536-ac99-4584882994d8 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:33.797593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 659/1312] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:33.803150 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee58079-b6d0-43e7-980c-98fbe496daa5 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:33.803682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee58079-b6d0-43e7-980c-98fbe496daa5 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:33.803781 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee58079-b6d0-43e7-980c-98fbe496daa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:33.896297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-4ee58079-b6d0-43e7-980c-98fbe496daa5 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:33.897969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-4ee58079-b6d0-43e7-980c-98fbe496daa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:34.009429 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 654/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:33:34.049382 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa900fef-a03d-4536-ac99-4584882994d8 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:34.050940 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-aa900fef-a03d-4536-ac99-4584882994d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:34.158255 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa900fef-a03d-4536-ac99-4584882994d8 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:34.236920 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26831ec6-d337-42e2-86f7-19b49e129384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:34.237890 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831ec6-d337-42e2-86f7-19b49e129384 service neutron] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:34.238410 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831ec6-d337-42e2-86f7-19b49e129384 service neutron] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:34.238858 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831ec6-d337-42e2-86f7-19b49e129384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:34.240843 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-26831ec6-d337-42e2-86f7-19b49e129384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:34.361820 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26831ec6-d337-42e2-86f7-19b49e129384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:34.363379 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26831ec6-d337-42e2-86f7-19b49e129384 service neutron] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:34.382826 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 660/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:33:36.987910 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7b41b-0fb5-4427-a11a-585526957f2c None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:36.988473 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7b41b-0fb5-4427-a11a-585526957f2c None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:36.988473 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7b41b-0fb5-4427-a11a-585526957f2c None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:36.989888 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 655/1315] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:36.995048 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec795ad2-52dd-4a86-8f9b-f9313f8e5ddf None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:36.995519 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec795ad2-52dd-4a86-8f9b-f9313f8e5ddf None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:36.995629 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec795ad2-52dd-4a86-8f9b-f9313f8e5ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:37.058776 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-ec795ad2-52dd-4a86-8f9b-f9313f8e5ddf None None] MFA Rules not processed for user `0fe86d9a6c224cacaa25f0d29d448f20`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:37.059915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-ec795ad2-52dd-4a86-8f9b-f9313f8e5ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:37.153390 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 661/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:37.192244 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7b41b-0fb5-4427-a11a-585526957f2c None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:37.193823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-c0d7b41b-0fb5-4427-a11a-585526957f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:37.293050 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7b41b-0fb5-4427-a11a-585526957f2c None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:37.362363 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585b564-89ee-466a-91d0-85ab61b3bb5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:37.362956 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b564-89ee-466a-91d0-85ab61b3bb5f service neutron] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:37.363242 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b564-89ee-466a-91d0-85ab61b3bb5f service neutron] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:37.363521 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b564-89ee-466a-91d0-85ab61b3bb5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:37.365179 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-7585b564-89ee-466a-91d0-85ab61b3bb5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:37.468440 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585b564-89ee-466a-91d0-85ab61b3bb5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:37.469899 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585b564-89ee-466a-91d0-85ab61b3bb5f service neutron] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:37.489190 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 656/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:33:40.269311 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a94ad-2b21-47df-af33-4a436443fbda None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:40.269869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a94ad-2b21-47df-af33-4a436443fbda None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:40.269869 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a94ad-2b21-47df-af33-4a436443fbda None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:40.271259 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 662/1318] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:40.276552 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f183df-0d38-4e23-b2be-27170ecd92c5 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:40.276952 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f183df-0d38-4e23-b2be-27170ecd92c5 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:40.277158 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f183df-0d38-4e23-b2be-27170ecd92c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:40.342044 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-63f183df-0d38-4e23-b2be-27170ecd92c5 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:40.343276 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-63f183df-0d38-4e23-b2be-27170ecd92c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:40.437177 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 657/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:40.479853 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a94ad-2b21-47df-af33-4a436443fbda None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:40.481448 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-ee0a94ad-2b21-47df-af33-4a436443fbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:40.583384 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a94ad-2b21-47df-af33-4a436443fbda None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:40.654397 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7128524-0cd6-45d9-a776-a103bd463d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:40.655048 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128524-0cd6-45d9-a776-a103bd463d8b service neutron] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:40.655325 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128524-0cd6-45d9-a776-a103bd463d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:40.655643 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128524-0cd6-45d9-a776-a103bd463d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:40.657287 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-e7128524-0cd6-45d9-a776-a103bd463d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:40.762521 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128524-0cd6-45d9-a776-a103bd463d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:40.763982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128524-0cd6-45d9-a776-a103bd463d8b service neutron] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:40.781386 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 663/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:33:43.660740 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d66736-bf21-4d7c-a3f4-df379aff8fd2 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:43.661308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d66736-bf21-4d7c-a3f4-df379aff8fd2 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:43.661433 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d66736-bf21-4d7c-a3f4-df379aff8fd2 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:43.662709 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 658/1321] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:43.668056 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeb0fe-4686-4c8b-96be-2426f3389d0f None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:43.668485 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeb0fe-4686-4c8b-96be-2426f3389d0f None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:43.668596 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeb0fe-4686-4c8b-96be-2426f3389d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:43.734642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-ddeeb0fe-4686-4c8b-96be-2426f3389d0f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:43.736001 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-ddeeb0fe-4686-4c8b-96be-2426f3389d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:43.832807 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 664/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:43.884446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1427a-5cd0-470a-bf17-fdd30c368efa None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:43.884734 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1427a-5cd0-470a-bf17-fdd30c368efa None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:43.885013 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1427a-5cd0-470a-bf17-fdd30c368efa None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:43.886426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 659/1323] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 9 15:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:43.893138 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e7a78-be17-4729-8b41-4182532eff49 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:43.893429 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e7a78-be17-4729-8b41-4182532eff49 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:43.893703 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e7a78-be17-4729-8b41-4182532eff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:43.967642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-8d4e7a78-be17-4729-8b41-4182532eff49 None None] MFA Rules not processed for user `5382d7219c244430ae6f37d6b32e7d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:43.968778 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-8d4e7a78-be17-4729-8b41-4182532eff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:44.042647 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 665/1324] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 9 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:33:44.051825 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1427a-5cd0-470a-bf17-fdd30c368efa None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:44.053425 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-93e1427a-5cd0-470a-bf17-fdd30c368efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:44.154471 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1427a-5cd0-470a-bf17-fdd30c368efa None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:44.226349 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d22952-2223-4b01-82a5-89b6516751c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:44.226932 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22952-2223-4b01-82a5-89b6516751c5 service neutron] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:44.227219 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22952-2223-4b01-82a5-89b6516751c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:44.227501 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22952-2223-4b01-82a5-89b6516751c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:44.229297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-24d22952-2223-4b01-82a5-89b6516751c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:44.336653 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d22952-2223-4b01-82a5-89b6516751c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:44.338136 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d22952-2223-4b01-82a5-89b6516751c5 service neutron] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:44.356907 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 660/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:33:46.897409 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea03f-2fc0-4ed3-9fa1-0813ce49a33b None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:46.897945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea03f-2fc0-4ed3-9fa1-0813ce49a33b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:46.898151 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea03f-2fc0-4ed3-9fa1-0813ce49a33b None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:46.899650 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 666/1326] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:46.904998 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b155-ee2b-454e-8918-f6f17f8718ec None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:46.905452 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b155-ee2b-454e-8918-f6f17f8718ec None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:46.905635 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98b155-ee2b-454e-8918-f6f17f8718ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:46.969390 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-db98b155-ee2b-454e-8918-f6f17f8718ec None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:46.970597 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-db98b155-ee2b-454e-8918-f6f17f8718ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:47.069217 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 661/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:47.111792 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ea03f-2fc0-4ed3-9fa1-0813ce49a33b None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:47.113368 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-574ea03f-2fc0-4ed3-9fa1-0813ce49a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:47.217316 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ea03f-2fc0-4ed3-9fa1-0813ce49a33b None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:47.288033 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e951b-d8c0-4010-a0c9-a614cacafd13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:47.288684 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e951b-d8c0-4010-a0c9-a614cacafd13 service neutron] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:47.288968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e951b-d8c0-4010-a0c9-a614cacafd13 service neutron] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:47.289275 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e951b-d8c0-4010-a0c9-a614cacafd13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:47.290934 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-c00e951b-d8c0-4010-a0c9-a614cacafd13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:47.415628 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e951b-d8c0-4010-a0c9-a614cacafd13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:47.417095 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e951b-d8c0-4010-a0c9-a614cacafd13 service neutron] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:47.435719 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 667/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:33:49.706067 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ca5a5-3573-4872-99e7-387b267503f1 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:49.706632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ca5a5-3573-4872-99e7-387b267503f1 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:49.706632 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ca5a5-3573-4872-99e7-387b267503f1 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:49.708028 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 662/1329] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:49.713323 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072269d-a0d7-4c7a-9bf4-510e7eef42fb None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:49.713749 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072269d-a0d7-4c7a-9bf4-510e7eef42fb None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:49.713900 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072269d-a0d7-4c7a-9bf4-510e7eef42fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:49.787272 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-2072269d-a0d7-4c7a-9bf4-510e7eef42fb None None] MFA Rules not processed for user `0fe86d9a6c224cacaa25f0d29d448f20`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:49.788542 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-2072269d-a0d7-4c7a-9bf4-510e7eef42fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:49.881758 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 668/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:49.919392 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ca5a5-3573-4872-99e7-387b267503f1 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:49.920950 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-630ca5a5-3573-4872-99e7-387b267503f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:50.021816 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ca5a5-3573-4872-99e7-387b267503f1 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:50.091523 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988858a-e729-4e08-8a1c-b2e15f00330d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:50.092175 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988858a-e729-4e08-8a1c-b2e15f00330d service neutron] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:50.092461 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988858a-e729-4e08-8a1c-b2e15f00330d service neutron] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:50.092738 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988858a-e729-4e08-8a1c-b2e15f00330d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:50.094427 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-e988858a-e729-4e08-8a1c-b2e15f00330d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:50.205399 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988858a-e729-4e08-8a1c-b2e15f00330d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:50.206847 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988858a-e729-4e08-8a1c-b2e15f00330d service neutron] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:50.223971 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 663/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:33:52.828113 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2b92c-3895-4180-9cc6-c5e6059c6f56 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:52.828658 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2b92c-3895-4180-9cc6-c5e6059c6f56 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:52.828658 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2b92c-3895-4180-9cc6-c5e6059c6f56 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:52.830082 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 669/1332] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:52.835432 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72769cb-a244-4457-b6cf-0b5d2d0a203e None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:52.835849 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72769cb-a244-4457-b6cf-0b5d2d0a203e None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:52.836007 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72769cb-a244-4457-b6cf-0b5d2d0a203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:52.898751 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-e72769cb-a244-4457-b6cf-0b5d2d0a203e None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:52.899958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-e72769cb-a244-4457-b6cf-0b5d2d0a203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:52.992048 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 664/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:53.027836 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2b92c-3895-4180-9cc6-c5e6059c6f56 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:53.029919 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-97c2b92c-3895-4180-9cc6-c5e6059c6f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:53.130273 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2b92c-3895-4180-9cc6-c5e6059c6f56 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:53.203757 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed34fc8-2eff-44d4-810c-7d8721f8d44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:53.204393 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34fc8-2eff-44d4-810c-7d8721f8d44c service neutron] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:53.204674 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34fc8-2eff-44d4-810c-7d8721f8d44c service neutron] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:53.204945 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed34fc8-2eff-44d4-810c-7d8721f8d44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:53.206604 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-0ed34fc8-2eff-44d4-810c-7d8721f8d44c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:53.309730 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34fc8-2eff-44d4-810c-7d8721f8d44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:53.311147 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed34fc8-2eff-44d4-810c-7d8721f8d44c service neutron] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:53.328016 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 670/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:33:55.577907 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f6808-107d-4dc9-ba95-a2bbcf481b33 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:55.578447 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f6808-107d-4dc9-ba95-a2bbcf481b33 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:55.578447 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f6808-107d-4dc9-ba95-a2bbcf481b33 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:55.579823 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 665/1335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:33:55.585057 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b9e22-f6aa-4512-92eb-6c72f6d0723b None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:55.585434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b9e22-f6aa-4512-92eb-6c72f6d0723b None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:55.585642 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b9e22-f6aa-4512-92eb-6c72f6d0723b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:55.647907 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-513b9e22-f6aa-4512-92eb-6c72f6d0723b None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:55.649118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-513b9e22-f6aa-4512-92eb-6c72f6d0723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:55.743829 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 671/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:55.783875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f6808-107d-4dc9-ba95-a2bbcf481b33 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:55.785444 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-5d5f6808-107d-4dc9-ba95-a2bbcf481b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:55.883900 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f6808-107d-4dc9-ba95-a2bbcf481b33 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:55.953759 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0e7d7-cc4f-4b0b-ac8c-eb2bcc493cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:55.954403 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0e7d7-cc4f-4b0b-ac8c-eb2bcc493cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:55.954693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0e7d7-cc4f-4b0b-ac8c-eb2bcc493cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:55.954980 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0e7d7-cc4f-4b0b-ac8c-eb2bcc493cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:55.956622 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-12a0e7d7-cc4f-4b0b-ac8c-eb2bcc493cb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:56.062302 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0e7d7-cc4f-4b0b-ac8c-eb2bcc493cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:56.063742 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0e7d7-cc4f-4b0b-ac8c-eb2bcc493cb0 service neutron] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:56.081682 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 666/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:33:58.630753 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9bc57-2848-47f0-a2d3-43ebea93cbdd None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:58.631283 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9bc57-2848-47f0-a2d3-43ebea93cbdd None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:58.631283 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9bc57-2848-47f0-a2d3-43ebea93cbdd None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:58.632597 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 672/1338] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:33:58.638628 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c46037-91f9-476b-a499-5d9481261895 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:58.639035 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c46037-91f9-476b-a499-5d9481261895 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:58.639207 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c46037-91f9-476b-a499-5d9481261895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:58.702339 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-22c46037-91f9-476b-a499-5d9481261895 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:33:58.703556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-22c46037-91f9-476b-a499-5d9481261895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:58.801813 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 667/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:33:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:33:58.836990 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9bc57-2848-47f0-a2d3-43ebea93cbdd None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:33:58.838546 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-fbe9bc57-2848-47f0-a2d3-43ebea93cbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:58.937771 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9bc57-2848-47f0-a2d3-43ebea93cbdd None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:33:59.006789 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cf6b3-0333-4f46-92b5-456567adbe6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:33:59.007426 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cf6b3-0333-4f46-92b5-456567adbe6f service neutron] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:33:59.007708 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cf6b3-0333-4f46-92b5-456567adbe6f service neutron] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:33:59.007982 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cf6b3-0333-4f46-92b5-456567adbe6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:33:59.009635 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-767cf6b3-0333-4f46-92b5-456567adbe6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:33:59.114150 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767cf6b3-0333-4f46-92b5-456567adbe6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:33:59.115593 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767cf6b3-0333-4f46-92b5-456567adbe6f service neutron] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:33:59.132738 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 673/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:33:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:34:03.384275 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe70c1-28c4-4b00-a0f6-d5e01943de6f None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:03.384860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe70c1-28c4-4b00-a0f6-d5e01943de6f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:03.384860 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe70c1-28c4-4b00-a0f6-d5e01943de6f None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:03.386170 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 668/1341] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:03.391605 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53541-1476-4cdd-8204-3cb9287182d6 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:03.392030 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53541-1476-4cdd-8204-3cb9287182d6 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:03.392168 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa53541-1476-4cdd-8204-3cb9287182d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:03.461702 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-caa53541-1476-4cdd-8204-3cb9287182d6 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:03.462844 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-caa53541-1476-4cdd-8204-3cb9287182d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:03.565501 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 674/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:03.604163 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe70c1-28c4-4b00-a0f6-d5e01943de6f None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:34:03.607079 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-bcbe70c1-28c4-4b00-a0f6-d5e01943de6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:03.707113 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe70c1-28c4-4b00-a0f6-d5e01943de6f None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:34:03.710707 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fdf79-c403-48d4-8b70-63cac55ab974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:34:03.711356 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fdf79-c403-48d4-8b70-63cac55ab974 service neutron] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:03.711644 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fdf79-c403-48d4-8b70-63cac55ab974 service neutron] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:03.711923 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fdf79-c403-48d4-8b70-63cac55ab974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:03.721345 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fdf79-c403-48d4-8b70-63cac55ab974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:34:03.722787 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fdf79-c403-48d4-8b70-63cac55ab974 service neutron] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:34:03.733014 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 669/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:34:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:34:06.615250 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fba52-dc44-4052-a528-1a076d6342c9 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:06.615797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fba52-dc44-4052-a528-1a076d6342c9 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:06.615797 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fba52-dc44-4052-a528-1a076d6342c9 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:06.617117 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 675/1344] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:06.623052 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c16577-77c7-4757-8b0a-20b7505fbef9 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:06.623491 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c16577-77c7-4757-8b0a-20b7505fbef9 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:06.623600 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c16577-77c7-4757-8b0a-20b7505fbef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:06.651237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-68c16577-77c7-4757-8b0a-20b7505fbef9 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:06.652426 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-68c16577-77c7-4757-8b0a-20b7505fbef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:06.730187 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 670/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:06.768815 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fba52-dc44-4052-a528-1a076d6342c9 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:34:06.781317 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fba52-dc44-4052-a528-1a076d6342c9 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:34:06.784894 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debe9ed-7ed0-4ffd-9338-f58037c9e413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:34:06.785567 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debe9ed-7ed0-4ffd-9338-f58037c9e413 service neutron] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:06.785852 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debe9ed-7ed0-4ffd-9338-f58037c9e413 service neutron] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:06.786128 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debe9ed-7ed0-4ffd-9338-f58037c9e413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:06.794915 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4debe9ed-7ed0-4ffd-9338-f58037c9e413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:34:06.796370 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4debe9ed-7ed0-4ffd-9338-f58037c9e413 service neutron] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:34:06.805588 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 676/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:34:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:34:11.062098 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8cf46-20ee-4196-9b04-7b4fa3d3da97 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:11.062617 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8cf46-20ee-4196-9b04-7b4fa3d3da97 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:11.062617 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8cf46-20ee-4196-9b04-7b4fa3d3da97 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:11.063966 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 671/1347] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:11.069302 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc301-1f5b-4b0b-a912-21b6f79f6c12 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:11.069810 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc301-1f5b-4b0b-a912-21b6f79f6c12 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:11.069922 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc301-1f5b-4b0b-a912-21b6f79f6c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:11.089156 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-36fbc301-1f5b-4b0b-a912-21b6f79f6c12 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:11.090968 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-36fbc301-1f5b-4b0b-a912-21b6f79f6c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:11.100186 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 677/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:34:11.137264 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8cf46-20ee-4196-9b04-7b4fa3d3da97 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:34:11.148054 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8cf46-20ee-4196-9b04-7b4fa3d3da97 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:34:11.151609 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f79ab-6892-401b-b18c-c6effb5ba89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5382d7219c244430ae6f37d6b32e7d8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:34:11.152905 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f79ab-6892-401b-b18c-c6effb5ba89c service neutron] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:11.153241 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f79ab-6892-401b-b18c-c6effb5ba89c service neutron] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:11.153521 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f79ab-6892-401b-b18c-c6effb5ba89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:11.162422 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f79ab-6892-401b-b18c-c6effb5ba89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:34:11.163900 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f79ab-6892-401b-b18c-c6effb5ba89c service neutron] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:34:11.173800 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 672/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:34:16.586754 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80ff89-f725-425c-993a-22fda44e8fb0 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:16.587608 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80ff89-f725-425c-993a-22fda44e8fb0 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:16.588044 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80ff89-f725-425c-993a-22fda44e8fb0 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:16.606292 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 678/1350] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:34:16 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:16.611446 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac12c42-aaaa-4585-849b-0c07134f00fd None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:16.612236 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac12c42-aaaa-4585-849b-0c07134f00fd None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:16.612670 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac12c42-aaaa-4585-849b-0c07134f00fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:16.776237 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-cac12c42-aaaa-4585-849b-0c07134f00fd None None] MFA Rules not processed for user `34415b5565ee42fea28ffb93825754b1`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:16.778984 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-cac12c42-aaaa-4585-849b-0c07134f00fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:16.881450 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 673/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:34:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:21.347035 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493336a5-095f-431b-9c94-c013ab1dcd52 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:21.347596 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493336a5-095f-431b-9c94-c013ab1dcd52 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:21.347911 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493336a5-095f-431b-9c94-c013ab1dcd52 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:21.349544 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 679/1352] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:34:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:21.354765 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ff0c-6752-4a95-a916-db8e0c518e12 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:21.355217 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ff0c-6752-4a95-a916-db8e0c518e12 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:21.355578 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8ff0c-6752-4a95-a916-db8e0c518e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:21.374004 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-0ab8ff0c-6752-4a95-a916-db8e0c518e12 None None] MFA Rules not processed for user `34415b5565ee42fea28ffb93825754b1`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:21.375329 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-0ab8ff0c-6752-4a95-a916-db8e0c518e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:21.384410 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 674/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:22.933484 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835faa85-fc31-4cde-a1fb-967ec697af3e None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:22.934034 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835faa85-fc31-4cde-a1fb-967ec697af3e None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:22.934209 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835faa85-fc31-4cde-a1fb-967ec697af3e None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:22.935846 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 680/1354] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:22.941743 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87776103-36de-4b4a-8aa9-e8ea7ccd78f7 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:22.942308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87776103-36de-4b4a-8aa9-e8ea7ccd78f7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:22.942308 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87776103-36de-4b4a-8aa9-e8ea7ccd78f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:22.961556 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-87776103-36de-4b4a-8aa9-e8ea7ccd78f7 None None] MFA Rules not processed for user `34415b5565ee42fea28ffb93825754b1`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:22.962804 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-87776103-36de-4b4a-8aa9-e8ea7ccd78f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:22.972160 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 675/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:26.684864 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1f3c2-8aec-4213-9f86-d0dcbe250f08 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:26.685459 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1f3c2-8aec-4213-9f86-d0dcbe250f08 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:26.685740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1f3c2-8aec-4213-9f86-d0dcbe250f08 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:26.687343 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 681/1356] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:26.692546 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2b569-5657-465b-bedb-886ee6c378c7 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:26.692958 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2b569-5657-465b-bedb-886ee6c378c7 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:26.693148 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a2b569-5657-465b-bedb-886ee6c378c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:26.711805 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-85a2b569-5657-465b-bedb-886ee6c378c7 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:26.713017 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-85a2b569-5657-465b-bedb-886ee6c378c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:26.721969 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 676/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:26.770689 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b44b0f-dcba-4325-a0e4-d103fb065fe5 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:26.771125 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b44b0f-dcba-4325-a0e4-d103fb065fe5 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:26.771515 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b44b0f-dcba-4325-a0e4-d103fb065fe5 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:26.772992 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 682/1358] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 9 15:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:26.779268 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa499-7b36-4f9a-b255-3ef166c99f25 None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:26.779639 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa499-7b36-4f9a-b255-3ef166c99f25 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:26.780092 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103fa499-7b36-4f9a-b255-3ef166c99f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:26.828093 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-103fa499-7b36-4f9a-b255-3ef166c99f25 None None] MFA Rules not processed for user `17730dd371184663b2c0eb091cc668a3`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:26.829420 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-103fa499-7b36-4f9a-b255-3ef166c99f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:26.909331 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 677/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 9 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:26.918452 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b44b0f-dcba-4325-a0e4-d103fb065fe5 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:34:26.934504 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b44b0f-dcba-4325-a0e4-d103fb065fe5 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:34:26.940434 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16889e25-6196-4486-9ce7-bf46c954d7f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17730dd371184663b2c0eb091cc668a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:34:26.941353 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16889e25-6196-4486-9ce7-bf46c954d7f5 service nova] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:26.941834 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16889e25-6196-4486-9ce7-bf46c954d7f5 service nova] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:26.942304 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16889e25-6196-4486-9ce7-bf46c954d7f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:26.963217 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16889e25-6196-4486-9ce7-bf46c954d7f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:34:26.965005 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16889e25-6196-4486-9ce7-bf46c954d7f5 service nova] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:34:26.975173 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 683/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:34:29.443846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfe8c8-97f9-407b-b7e1-4dd26a269618 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:29.444471 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfe8c8-97f9-407b-b7e1-4dd26a269618 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:29.444471 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfe8c8-97f9-407b-b7e1-4dd26a269618 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:29.445991 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 678/1361] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:29.452621 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14b202-508e-48f6-9be4-6acd733ff3d2 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:29.453104 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14b202-508e-48f6-9be4-6acd733ff3d2 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:29.453673 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14b202-508e-48f6-9be4-6acd733ff3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:29.472532 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-0c14b202-508e-48f6-9be4-6acd733ff3d2 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:29.474353 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-0c14b202-508e-48f6-9be4-6acd733ff3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:29.482740 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 684/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:34:29.528897 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfe8c8-97f9-407b-b7e1-4dd26a269618 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:34:29.543710 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfe8c8-97f9-407b-b7e1-4dd26a269618 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:34:29.549663 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab08e32-c38c-497f-8caa-88f28575c766 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17730dd371184663b2c0eb091cc668a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:34:29.550334 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab08e32-c38c-497f-8caa-88f28575c766 service nova] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:29.550599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab08e32-c38c-497f-8caa-88f28575c766 service nova] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:29.550875 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab08e32-c38c-497f-8caa-88f28575c766 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:29.559658 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab08e32-c38c-497f-8caa-88f28575c766 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:34:29.561297 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab08e32-c38c-497f-8caa-88f28575c766 service nova] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:34:29.571172 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 679/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 15:34:32.125656 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d62a8-b74a-428a-a1b2-216d547e3eed None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:32.126206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d62a8-b74a-428a-a1b2-216d547e3eed None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:32.126206 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d62a8-b74a-428a-a1b2-216d547e3eed None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:32.127619 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 685/1364] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:32.133319 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be3dc4-f717-453a-b8a2-aaffc4adab5f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:32.133775 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be3dc4-f717-453a-b8a2-aaffc4adab5f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:32.134039 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be3dc4-f717-453a-b8a2-aaffc4adab5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:32.151567 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-04be3dc4-f717-453a-b8a2-aaffc4adab5f None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:32.153333 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-04be3dc4-f717-453a-b8a2-aaffc4adab5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:32.161494 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 680/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:32.206908 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fc071-c84c-48da-a9d2-f6dc18337104 None None] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:32.207331 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fc071-c84c-48da-a9d2-f6dc18337104 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:32.207796 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fc071-c84c-48da-a9d2-f6dc18337104 None None] PATH_INFO: `/` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:32.209336 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 686/1366] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 9 15:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:32.215307 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec30cb-ebd6-4ca2-a53f-ef3853280a2f None None] REQUEST_METHOD: `POST` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:32.215603 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec30cb-ebd6-4ca2-a53f-ef3853280a2f None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:32.215883 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec30cb-ebd6-4ca2-a53f-ef3853280a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:32.235073 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.auth.core [None req-81ec30cb-ebd6-4ca2-a53f-ef3853280a2f None None] MFA Rules not processed for user `17730dd371184663b2c0eb091cc668a3`. Rule list: `[]` (Enabled: `True`). {{(pid=206281) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:32.236599 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.fernet_utils [None req-81ec30cb-ebd6-4ca2-a53f-ef3853280a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206281) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:32.246360 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 681/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 9 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:32.254126 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fc071-c84c-48da-a9d2-f6dc18337104 None None] Authenticating user token {{(pid=206280) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:34:32.268980 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fc071-c84c-48da-a9d2-f6dc18337104 None None] Validating token access rules against request {{(pid=206280) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:34:32.275076 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f9928-aecc-474c-9d30-43fd9ae11af6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17730dd371184663b2c0eb091cc668a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206280) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:34:32.275763 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f9928-aecc-474c-9d30-43fd9ae11af6 service nova] REQUEST_METHOD: `GET` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:32.276047 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f9928-aecc-474c-9d30-43fd9ae11af6 service nova] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:32.276325 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f9928-aecc-474c-9d30-43fd9ae11af6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:32.285096 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f9928-aecc-474c-9d30-43fd9ae11af6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:34:32.286996 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f9928-aecc-474c-9d30-43fd9ae11af6 service nova] RBAC: Authorization granted {{(pid=206280) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:34:32.297137 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 687/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 15:34:33.121382 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b61a-1612-46ad-ae2e-c7576670d1f1 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:33.121730 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b61a-1612-46ad-ae2e-c7576670d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:33.121970 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b61a-1612-46ad-ae2e-c7576670d1f1 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:33.123868 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 682/1369] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:33.130050 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d7492-d4b2-425f-9b60-cce6840b3729 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:33.130583 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d7492-d4b2-425f-9b60-cce6840b3729 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:33.130666 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d7492-d4b2-425f-9b60-cce6840b3729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:33.149859 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-dc1d7492-d4b2-425f-9b60-cce6840b3729 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:33.151110 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-dc1d7492-d4b2-425f-9b60-cce6840b3729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:33.160218 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 688/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:34:34.640768 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46219da-2c14-4253-b878-9fdabf90ff51 None None] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:34.641395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46219da-2c14-4253-b878-9fdabf90ff51 None None] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:34.641395 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46219da-2c14-4253-b878-9fdabf90ff51 None None] PATH_INFO: `/` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:34.643310 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 683/1371] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:34:34.648513 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a993a6d-0413-4c9a-8b43-6313e744a592 None None] REQUEST_METHOD: `POST` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:34.648981 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a993a6d-0413-4c9a-8b43-6313e744a592 None None] SCRIPT_NAME: `/identity` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:34.649118 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a993a6d-0413-4c9a-8b43-6313e744a592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=206280) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:34.666511 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.auth.core [None req-1a993a6d-0413-4c9a-8b43-6313e744a592 None None] MFA Rules not processed for user `05c2f4bd321b4a50a016143bb5412e00`. Rule list: `[]` (Enabled: `True`). {{(pid=206280) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:34:34.668266 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: DEBUG keystone.common.fernet_utils [None req-1a993a6d-0413-4c9a-8b43-6313e744a592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=206280) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:34:34.675979 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: [pid: 206280|app: 0|req: 689/1372] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:34:34.718869 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46219da-2c14-4253-b878-9fdabf90ff51 None None] Authenticating user token {{(pid=206281) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:34:34.733605 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46219da-2c14-4253-b878-9fdabf90ff51 None None] Validating token access rules against request {{(pid=206281) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:34:34.739580 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d07e4-0596-4c0f-89ed-96e4dc304fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17730dd371184663b2c0eb091cc668a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5d4f68f7dc4464a901b1d2d6b617c68', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=206281) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:34:34.740198 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d07e4-0596-4c0f-89ed-96e4dc304fb2 service nova] REQUEST_METHOD: `GET` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:34:34.740512 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d07e4-0596-4c0f-89ed-96e4dc304fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:34:34.740776 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d07e4-0596-4c0f-89ed-96e4dc304fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=206281) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:34:34.749846 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d07e4-0596-4c0f-89ed-96e4dc304fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:34:34.751438 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d07e4-0596-4c0f-89ed-96e4dc304fb2 service nova] RBAC: Authorization granted {{(pid=206281) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:34:34.760693 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: [pid: 206281|app: 0|req: 684/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:34:57.040313 dv-cinder-840678-3-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 09 15:34:57.040860 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: Mon May 9 15:34:57 2022 - graceful shutdown triggered... May 09 15:34:57.041456 dv-cinder-840678-3-iscsi devstack@keystone.service[206280]: Gracefully killing worker 1 (pid: 206280)... May 09 15:34:57.041854 dv-cinder-840678-3-iscsi devstack@keystone.service[206281]: Gracefully killing worker 2 (pid: 206281)... May 09 15:34:58.042509 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: worker 1 buried after 1 seconds May 09 15:34:58.042509 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: worker 2 buried after 1 seconds May 09 15:34:58.042509 dv-cinder-840678-3-iscsi devstack@keystone.service[206268]: goodbye to uWSGI. May 09 15:34:58.044088 dv-cinder-840678-3-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 09 15:34:58.044633 dv-cinder-840678-3-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 09 15:40:20.937174 dv-cinder-840678-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 09 15:40:20.946494 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 9 15:40:20 2022] *** May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: nodename: dv-cinder-840678-3-iscsi May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: machine: x86_64 May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: clock source: unix May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: pcre jit disabled May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: detected number of CPU cores: 4 May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: current working directory: / May 09 15:40:20.950098 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: detected binary path: /usr/bin/uwsgi-core May 09 15:40:20.950872 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: your processes number limit is 47680 May 09 15:40:20.950872 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: your memory page size is 4096 bytes May 09 15:40:20.950872 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: detected max file descriptor number: 2048 May 09 15:40:20.950872 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: lock engine: pthread robust mutexes May 09 15:40:20.950872 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: thunder lock: enabled May 09 15:40:20.950872 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 09 15:40:20.950872 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 09 15:40:20.999205 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: Python main interpreter initialized at 0x55cda59809c0 May 09 15:40:20.999205 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: python threads support enabled May 09 15:40:20.999205 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: your server socket listen backlog is limited to 100 connections May 09 15:40:20.999205 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: your mercy for graceful operations on workers is 90 seconds May 09 15:40:20.999666 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: mapped 403077 bytes (393 KB) for 2 cores May 09 15:40:20.999666 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: *** Operational MODE: preforking *** May 09 15:40:20.999914 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: *** uWSGI is running in multiple interpreter mode *** May 09 15:40:20.999914 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: spawned uWSGI master process (pid: 275699) May 09 15:40:21.000126 dv-cinder-840678-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 09 15:40:21.000752 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: spawned uWSGI worker 1 (pid: 275711, cores: 1) May 09 15:40:21.005063 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: spawned uWSGI worker 2 (pid: 275712, cores: 1) May 09 15:40:21.005063 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 09 15:40:26.787889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:40:26.787889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: from cryptography.utils import int_from_bytes May 09 15:40:26.788804 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:40:26.788804 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: from cryptography.utils import int_from_bytes May 09 15:40:26.789356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:40:26.789356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: from cryptography.utils import int_from_bytes May 09 15:40:26.789805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 09 15:40:26.789805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: from cryptography.utils import int_from_bytes May 09 15:40:26.837938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 15:40:26.838132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 15:40:26.838312 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] command line args: [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 15:40:26.838424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 09 15:40:26.838521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 09 15:40:26.838618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 15:40:26.838710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] command line args: [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 09 15:40:26.838800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ================================================================================ {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 15:40:26.838952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 09 15:40:26.839065 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] admin_token = **** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.839162 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ================================================================================ {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 09 15:40:26.839254 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] config_dir = [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.839379 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] admin_token = **** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.839487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.839579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] config_source = [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.839704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] config_dir = [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.839800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] debug = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.839892 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.839983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.840267 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] config_source = [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.840267 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] debug = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.840504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.840631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.840779 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] insecure_debug = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.840916 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.841070 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.841220 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] insecure_debug = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.841337 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.841475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.841598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] list_limit = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.841726 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.841849 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] log_config_append = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.841968 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] list_limit = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842088 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842203 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] log_config_append = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842346 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] log_dir = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842538 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] log_file = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842630 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] log_dir = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842772 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842869 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] log_file = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.842968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843078 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843412 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843779 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.843961 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.844061 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.844158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.844305 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.844402 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.844493 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.844598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.844833 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.844932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845150 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845243 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845343 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845535 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] notification_format = cadf {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845909 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] notification_format = cadf {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.845909 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] publish_errors = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846486 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] publish_errors = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846668 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846758 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.846898 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847286 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847377 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] standard_threads = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847468 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847557 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] strict_password_check = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] standard_threads = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] strict_password_check = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847934 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] use_eventlog = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.847934 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] use_journal = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.848106 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.848213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] use_json = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.848305 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] use_eventlog = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.848395 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] use_stderr = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.848480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] use_journal = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.848568 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] use_syslog = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.848676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] use_json = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.848676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] use_stderr = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.849257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] watch_log_file = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.849348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] use_syslog = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.849439 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.849523 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] watch_log_file = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 09 15:40:26.849612 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.849706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.850303 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.850402 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.850487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.850577 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.850674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.850770 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.850861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851002 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851247 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851502 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851746 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851846 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.851997 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.852106 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.852237 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.852341 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.852484 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.852583 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.852719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.852817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.852966 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.853103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.853236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.853338 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.853474 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.853573 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.853727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.853827 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.853971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.854071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.854216 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.854321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.854461 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.854571 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.854716 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.854826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.854959 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.855063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.855217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.855312 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.855465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.855564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.855702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.855808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.855938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856038 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856172 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856410 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856517 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856643 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856884 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.856985 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.857181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.857298 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.857393 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.857485 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.857614 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.857715 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.857849 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.857960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.858091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.858190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.858340 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.858450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.858573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.858679 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.858813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.858925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859049 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859381 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859620 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859843 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.859973 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.860088 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.860190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.860288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.860425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.860527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.860667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.860786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.860906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861017 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861275 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861377 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.861945 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.862077 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.862181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.862318 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.862419 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.862550 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.862649 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.862785 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.862884 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863023 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863122 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.863962 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.864061 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.864196 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.864296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.864434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.864535 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.864680 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.864788 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.864916 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.865071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.865186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.865279 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.865406 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866145 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866609 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866700 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866890 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.866976 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.867140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.867235 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.867379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.867480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.867619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.867719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.867859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.867973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.868096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.868219 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.enabled = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.868327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.868463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.868687 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.enabled = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.868796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.868934 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.869086 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.869195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.869289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.869422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.869518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.869663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.869763 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.869902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870014 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870142 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870241 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870629 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.870966 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.871117 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.871218 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.871365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.871466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.871605 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.871704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.871843 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.872227 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.872379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.872483 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.872576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.872683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.872787 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.872880 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.872983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873102 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873407 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873604 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873699 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873883 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.873974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874156 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874251 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.874978 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.875100 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] auth.external = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.875205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.875337 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] auth.mapped = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.875447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.875590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] auth.external = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.875682 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.875829 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] auth.mapped = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.875933 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.876081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] auth.password = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.876187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.876317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] auth.token = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.876431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.876561 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.876662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] auth.password = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.876801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] auth.token = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.876901 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] catalog.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.877075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.877188 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.877283 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.877368 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] catalog.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.877537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.877638 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.877771 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.877873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878107 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878248 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] credential.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878482 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878585 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] credential.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] credential.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878830 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.878974 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.879073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] credential.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.879214 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.879318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.879460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.879560 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.879698 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.879797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.879934 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.880034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.880172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.880266 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.880438 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.880532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.880694 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.880818 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.881059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.881166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.881261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.881354 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.881507 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.881615 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.881760 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.881861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] federation.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882609 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] federation.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882840 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] federation.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.882985 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.883080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.883204 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.883298 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] federation.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.883450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.883552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.883692 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.883793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.883933 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.884038 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.884168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.884269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.884425 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.884519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.884666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.884767 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.884915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.885011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.885181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.885283 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.885431 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.885540 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.885670 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.885771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.885906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886000 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886146 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886257 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886389 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886716 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.886964 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.887096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.887203 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.887375 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.887465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.887579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.887678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.887814 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.887908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.888048 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.888147 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.888283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.888375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.888503 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.888604 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.888753 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.888854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.889004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.889159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.889267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.889374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.889526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.889627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.889762 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.889862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890003 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890107 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890237 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890579 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890820 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.890951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.891052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.891184 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.891275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.891425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.891528 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.891671 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.891771 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.891906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892001 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.892982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.893128 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.893222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.893367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.893461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.893607 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.893715 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.893840 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.893935 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.password = **** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.894076 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.894176 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.894307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.password = **** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.894409 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.894549 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.894647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.894786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.894879 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895258 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895363 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895835 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.895961 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.896062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.896211 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.896312 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.896443 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.896545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.896701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.896795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.896940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.897033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.897201 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.897309 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.897435 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.897541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.897670 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.897772 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.897906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898759 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898899 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.898998 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.899140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.899235 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.899374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.899475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.899613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.899714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.899846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.899938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.900081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.900190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.900315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.900416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.900553 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.900652 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.900789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.900978 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.901109 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.901214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.901307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.901406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.901633 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.901759 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.901855 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902101 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902247 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902385 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.902990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.903111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.903231 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.903352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.903466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.903580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.903718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.903839 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.903971 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] policy.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.904086 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.904199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.904311 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.904445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.904577 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] policy.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.904699 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.904813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.904927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] receipt.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.905108 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.905258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.905375 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.905487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.905601 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] receipt.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.905714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.905827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.905948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.906069 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.906189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.906303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.906422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] resource.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.906543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.906665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.906794 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.906915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] resource.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907049 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] resource.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907306 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907406 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] resource.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907586 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907670 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907784 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] revoke.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907878 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.907976 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908230 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] revoke.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] role.cache_time = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908403 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908500 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] role.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908676 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] role.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908770 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] role.cache_time = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.908896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] role.list_limit = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.909004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] role.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.909171 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.909279 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] role.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.909425 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.909526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] role.list_limit = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.909676 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.909770 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.909960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910152 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910895 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.910988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.911126 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.911230 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.911441 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.911592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.911592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.911778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.911879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.911989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.912076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.912173 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.912851 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.913179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.913311 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.913441 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.913556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.913681 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.913796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.913917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.914052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.914180 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.914296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.914409 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.914521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.914641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.914755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.914883 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.915006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.915132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.915260 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.915390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.915520 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.915645 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.915760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.915887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916599 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916730 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.916972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.917242 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.917341 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.917427 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.917527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] token.cache_time = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.917638 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.917734 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] token.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.917820 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.917911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] token.cache_time = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918104 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] token.provider = fernet {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918199 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] token.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918286 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918370 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918454 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918544 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] token.provider = fernet {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918728 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.918991 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.919110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.919205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.919356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.919459 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.919608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] trust.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.919709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.919847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.919941 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.920089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.920197 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] trust.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.920326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.920434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.920572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.920667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.920813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.920923 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.921071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.921209 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.921323 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.921436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.921559 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.921720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.921836 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.921949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.connection = **** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.922069 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.922195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.922597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.922734 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.connection = **** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.922851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.922979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.923105 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.923220 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.923333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.923453 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.923574 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.923701 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.923823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.923944 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.924058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.924179 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.924302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.924417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.924536 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.924650 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.924769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.924883 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.925009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.925214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.925333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.925764 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.925891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.926013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.926137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.926259 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.926380 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.926495 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.926615 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.926747 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.926875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.927018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.927314 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=275711) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 15:40:26.927431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.927431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 09 15:40:26.927931 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=275712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 09 15:40:26.959859 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.960159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.960310 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.961707 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.961946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.962176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.969245 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.970951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.998801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:26.999036 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.000295 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.000532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.052416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.052582 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.104799 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.105004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.105159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.105259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.171179 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.171414 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.171522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.171647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.171752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.171877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.171979 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.172104 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.172215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.172310 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.172404 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.172531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.195160 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.195297 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.195471 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.195589 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.195685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.195788 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.195882 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.195981 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.196079 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.196172 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.196265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.196358 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.196457 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.196544 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.207709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.207875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.222451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.222689 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.222803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.222910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.223021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.223115 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.223208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.223342 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.223446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.223550 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.223654 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.224004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=275712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.224108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.224302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=275711) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 09 15:40:27.241668 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.241797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.241897 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:40:27.241998 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:40:27.242216 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.242319 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.242414 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:40:27.242507 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 09 15:40:27.242683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.242792 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.242948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.243051 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.243208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.243312 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.243466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.243568 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.254453 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.254563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.254733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.254836 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.255006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.255121 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.255292 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.255396 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.263410 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.263528 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.266703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.266822 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.266984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.267086 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.267264 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.267366 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.267655 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.267766 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.267936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.268039 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.268197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.268300 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.268468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.268588 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.268744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.268853 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.269021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.269153 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.269316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.269418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.269585 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.269687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.297411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.297524 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.299034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.299137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.299317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.299421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.304751 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.304856 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.304986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.305114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.305289 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.305385 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.312633 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.312741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.312919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.313015 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.317598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.317711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.317889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.317989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.318169 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.318283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.318450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.318552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.318749 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.318860 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.319021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.319124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.319288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.319461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.319571 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.319665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.338651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.338846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.338960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.339126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.343436 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.343656 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.347055 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.347313 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.347416 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.347609 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.353122 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.353424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.356659 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.356988 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.360384 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.360687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.360789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.360976 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.361098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.361289 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.361392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.361569 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.361671 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.361846 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.361955 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.362130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.380794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.381208 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.381330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.381478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.381638 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.381737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.381831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.382102 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.389617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.390052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.391325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.391602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.391774 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.392055 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.394673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.394956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.395162 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.395259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.395448 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.395556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.395739 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.395836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.396022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.396300 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.409989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.410274 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.410376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.410552 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.410649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.410847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.410964 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.411127 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.411232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.411410 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.411523 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.411701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.411798 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.411961 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.430389 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.430639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.433778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.433973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.437088 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.437313 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.438693 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.438941 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.440326 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.440580 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.440683 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.440861 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.440963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.441196 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.448871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.449180 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.450491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.450804 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.453743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.454037 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.454132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.454322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.454418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.454598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.454711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.454879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.464340 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.464645 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.464747 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.464929 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.465031 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.465300 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.472863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=275711) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.473168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.473282 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=275712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 09 15:40:27.473449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.473560 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.473972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.474084 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.474246 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.474356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.474527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.474624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.474822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.474926 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.475101 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.475205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.475378 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.475473 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.475647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.475756 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.475916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.476019 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.476185 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.476285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.476456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.476562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.476716 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=275711) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.476818 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.476976 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=275712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 09 15:40:27.515838 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cda59809c0 pid: 275711 (default app) May 09 15:40:27.516060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cda59809c0 pid: 275712 (default app) May 09 15:40:27.519226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ebcdf-64bd-42be-9634-c9ef7f8afae1 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:27.519532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ebcdf-64bd-42be-9634-c9ef7f8afae1 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:27.519822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ebcdf-64bd-42be-9634-c9ef7f8afae1 None None] PATH_INFO: `/v3` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:27.524585 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon May 9 15:40:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:35.500512 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc8ec3-61e6-44bb-b7b8-bbefa1737096 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:35.501116 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc8ec3-61e6-44bb-b7b8-bbefa1737096 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:35.501284 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc8ec3-61e6-44bb-b7b8-bbefa1737096 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:35.505762 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:35.511610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:35.512014 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:35.512181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:35.544451 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG oslo_db.sqlalchemy.engines [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=275711) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 15:40:35.729124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG passlib.handlers.bcrypt [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=275711) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 15:40:35.729546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG passlib.handlers.bcrypt [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=275711) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 15:40:35.784303 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:35.785889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:35.908487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 09 15:40:35.932098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc8ec3-61e6-44bb-b7b8-bbefa1737096 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:35.966936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG oslo_db.sqlalchemy.engines [None req-00fc8ec3-61e6-44bb-b7b8-bbefa1737096 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=275712) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 09 15:40:36.096734 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc8ec3-61e6-44bb-b7b8-bbefa1737096 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:36.101313 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42a2c0-e535-482c-adb9-f46d8fa3de7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c3add198476495285e8aab3267a1d15', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:36.101974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a2c0-e535-482c-adb9-f46d8fa3de7b admin admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:36.102263 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a2c0-e535-482c-adb9-f46d8fa3de7b admin admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:36.102543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a2c0-e535-482c-adb9-f46d8fa3de7b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:36.110583 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42a2c0-e535-482c-adb9-f46d8fa3de7b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:36.161194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5e42a2c0-e535-482c-adb9-f46d8fa3de7b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 15:40:36.161538 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42a2c0-e535-482c-adb9-f46d8fa3de7b admin admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:36.172015 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 9 15:40:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 15:40:36.178061 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:36.196421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455064-7ee1-4522-9329-d2ce500569e6 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:36.200928 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb663c6-15bd-4659-924f-ed9e19861c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c3add198476495285e8aab3267a1d15', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:36.202068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb663c6-15bd-4659-924f-ed9e19861c9b admin admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:36.202362 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb663c6-15bd-4659-924f-ed9e19861c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:36.202644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb663c6-15bd-4659-924f-ed9e19861c9b admin admin] PATH_INFO: `/v3/projects/3c3add198476495285e8aab3267a1d15` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:36.204186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb663c6-15bd-4659-924f-ed9e19861c9b admin admin] RBAC: Authorizing `identity:get_project(project_id=3c3add198476495285e8aab3267a1d15)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:36.253491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ebb663c6-15bd-4659-924f-ed9e19861c9b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 09 15:40:36.253845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb663c6-15bd-4659-924f-ed9e19861c9b admin admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:36.259068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon May 9 15:40:36 2022] GET /identity/v3/projects/3c3add198476495285e8aab3267a1d15 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:38.486445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2289-c5c2-43fa-bee3-57166252c4d0 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:38.487073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2289-c5c2-43fa-bee3-57166252c4d0 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:38.487301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2289-c5c2-43fa-bee3-57166252c4d0 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:38.492616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:38.498364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c22ea7-9634-4c3e-85f4-96da7dc3b295 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:38.498803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c22ea7-9634-4c3e-85f4-96da7dc3b295 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:38.498951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c22ea7-9634-4c3e-85f4-96da7dc3b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:38.518974 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-81c22ea7-9634-4c3e-85f4-96da7dc3b295 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:38.520536 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-81c22ea7-9634-4c3e-85f4-96da7dc3b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:38.529090 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 09 15:40:38.551757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf2289-c5c2-43fa-bee3-57166252c4d0 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:38.563631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf2289-c5c2-43fa-bee3-57166252c4d0 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:38.567855 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde2c771-75e3-4b89-8966-4162c9d32301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c3add198476495285e8aab3267a1d15', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:38.568487 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c771-75e3-4b89-8966-4162c9d32301 admin admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:38.568777 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c771-75e3-4b89-8966-4162c9d32301 admin admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:38.569090 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c771-75e3-4b89-8966-4162c9d32301 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:38.576648 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2c771-75e3-4b89-8966-4162c9d32301 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:38.578457 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde2c771-75e3-4b89-8966-4162c9d32301 admin admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:38.587583 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 9 15:40:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 09 15:40:38.592909 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c22ea7-9634-4c3e-85f4-96da7dc3b295 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:38.602603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c22ea7-9634-4c3e-85f4-96da7dc3b295 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:38.607090 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a2730-ebc1-4514-835c-77f3236a0d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c3add198476495285e8aab3267a1d15', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:38.607952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a2730-ebc1-4514-835c-77f3236a0d15 admin admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:38.608250 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a2730-ebc1-4514-835c-77f3236a0d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:38.608538 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a2730-ebc1-4514-835c-77f3236a0d15 admin admin] PATH_INFO: `/v3/users/d2a1baaf61604444b2c20a19956c1591` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:38.609973 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a2730-ebc1-4514-835c-77f3236a0d15 admin admin] RBAC: Authorizing `identity:get_user(user_id=d2a1baaf61604444b2c20a19956c1591)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:38.611834 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a2730-ebc1-4514-835c-77f3236a0d15 admin admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:38.613460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon May 9 15:40:38 2022] GET /identity/v3/users/d2a1baaf61604444b2c20a19956c1591 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:40.900806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd264e-a0d8-4202-8d3b-afb4ef37cbdd None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:40.904226 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd264e-a0d8-4202-8d3b-afb4ef37cbdd None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:40.904226 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd264e-a0d8-4202-8d3b-afb4ef37cbdd None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:40.904226 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:40:40.909005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee54003-5ede-40fa-b642-2b25cc192a00 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:40.909555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee54003-5ede-40fa-b642-2b25cc192a00 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:40.909661 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee54003-5ede-40fa-b642-2b25cc192a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:40.941294 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583777a9-5033-4b7d-9bf6-203320e80160 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:40.941592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583777a9-5033-4b7d-9bf6-203320e80160 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:40.941884 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583777a9-5033-4b7d-9bf6-203320e80160 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:40.943297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:40:40.949371 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:40.949670 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:40.949958 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:40.992584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG passlib.handlers.bcrypt [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=275712) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 09 15:40:40.993009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG passlib.handlers.bcrypt [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=275712) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 09 15:40:41.036632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-2ee54003-5ede-40fa-b642-2b25cc192a00 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:41.038425 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-2ee54003-5ede-40fa-b642-2b25cc192a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:41.040499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:41.042137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:41.117281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:40:41.140164 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee54003-5ede-40fa-b642-2b25cc192a00 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:41.140962 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:40:41.162266 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:41.236658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee54003-5ede-40fa-b642-2b25cc192a00 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:41.258518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77f6d3-9eed-4c71-be7b-f0dce0339153 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:41.269632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1dfc-d219-4707-b871-5a0cc45fae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:41.270443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1dfc-d219-4707-b871-5a0cc45fae80 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:41.270738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1dfc-d219-4707-b871-5a0cc45fae80 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:41.271023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f1dfc-d219-4707-b871-5a0cc45fae80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:41.282501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f1dfc-d219-4707-b871-5a0cc45fae80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:40:41.291264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef39d83-f763-4f49-8c85-9bcf68839e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:41.292157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef39d83-f763-4f49-8c85-9bcf68839e3e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:41.292447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef39d83-f763-4f49-8c85-9bcf68839e3e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:41.292728 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef39d83-f763-4f49-8c85-9bcf68839e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-c82f1dfc-d219-4707-b871-5a0cc45fae80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:40:41.294179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:41.295514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:40:41.296616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:40:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:40:41.301189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1dfc-d219-4707-b871-5a0cc45fae80 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:41.357703 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef39d83-f763-4f49-8c85-9bcf68839e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:41.359223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef39d83-f763-4f49-8c85-9bcf68839e3e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:41.396025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f1dfc-d219-4707-b871-5a0cc45fae80 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:41.429536 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:41.430145 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:41.430437 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:41.430722 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:41.431484 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:41.431799 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:41.433238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:41.444803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:41.450295 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:41.451986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:40:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:40:41.457221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef39d83-f763-4f49-8c85-9bcf68839e3e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:41.546961 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a8806-413e-47f3-bb6f-350b87a3223d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:41.552326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef39d83-f763-4f49-8c85-9bcf68839e3e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:41.581539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17841a-00fa-453a-afe2-cb003e072652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:41.582417 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17841a-00fa-453a-afe2-cb003e072652 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:41.582706 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17841a-00fa-453a-afe2-cb003e072652 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:41.582989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17841a-00fa-453a-afe2-cb003e072652 None admin] PATH_INFO: `/v3/users/d2a1baaf61604444b2c20a19956c1591` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:41.584204 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17841a-00fa-453a-afe2-cb003e072652 None admin] RBAC: Authorizing `identity:get_user(user_id=d2a1baaf61604444b2c20a19956c1591)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:41.585215 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:41.586048 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:41.586335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:41.586439 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17841a-00fa-453a-afe2-cb003e072652 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:41.586615 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:41.588179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:40:41 2022] GET /identity/v3/users/d2a1baaf61604444b2c20a19956c1591 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:41.593931 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17841a-00fa-453a-afe2-cb003e072652 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:41.696650 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17841a-00fa-453a-afe2-cb003e072652 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:41.717747 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:41.719216 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:41.730488 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f54506-9eb9-4b8e-b1cc-bd9722da6be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:41.731288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54506-9eb9-4b8e-b1cc-bd9722da6be7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:41.731578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54506-9eb9-4b8e-b1cc-bd9722da6be7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:41.731860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54506-9eb9-4b8e-b1cc-bd9722da6be7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:41.733094 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f54506-9eb9-4b8e-b1cc-bd9722da6be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:41.734532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f54506-9eb9-4b8e-b1cc-bd9722da6be7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:41.736157 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:40:41.740127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:41.741249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:41.742357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:40:41.742357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:40:41.742357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:40:41.742357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:40:41.742357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:40:41.742691 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f54506-9eb9-4b8e-b1cc-bd9722da6be7 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:41.748555 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:41.836415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f54506-9eb9-4b8e-b1cc-bd9722da6be7 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:41.844505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec0565-7e53-4930-b933-dfaa190db5fa None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:41.868580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be0877-dd02-4d2a-ad8a-249e457a1dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:41.869265 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be0877-dd02-4d2a-ad8a-249e457a1dba None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:41.869556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be0877-dd02-4d2a-ad8a-249e457a1dba None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:41.869839 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be0877-dd02-4d2a-ad8a-249e457a1dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:41.870682 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be0877-dd02-4d2a-ad8a-249e457a1dba None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.domain.id=default, user.id=d2a1baaf61604444b2c20a19956c1591) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:41.871003 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be0877-dd02-4d2a-ad8a-249e457a1dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:41.872455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be0877-dd02-4d2a-ad8a-249e457a1dba None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:41.877304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d14629-303e-49e4-b0dd-c32dd9fce3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:41.877966 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14629-303e-49e4-b0dd-c32dd9fce3f1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:41.878257 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14629-303e-49e4-b0dd-c32dd9fce3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:41.878540 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14629-303e-49e4-b0dd-c32dd9fce3f1 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:41.879105 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 9 15:40:41 2022] GET /identity/v3/role_assignments?user.id=d2a1baaf61604444b2c20a19956c1591&scope.domain.id=default&role.id=ebff922955b74c919d58613875326151 => generated 233 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:41.879310 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14629-303e-49e4-b0dd-c32dd9fce3f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:41.879581 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14629-303e-49e4-b0dd-c32dd9fce3f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:41.881004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14629-303e-49e4-b0dd-c32dd9fce3f1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:41.901800 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:44.456961 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce292e9-4dc6-4977-8510-dccfa3a80806 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:44.458067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce292e9-4dc6-4977-8510-dccfa3a80806 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:44.458623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce292e9-4dc6-4977-8510-dccfa3a80806 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:44.460908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:44.467107 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62212ae8-9057-4803-8eb8-ed5ad81ab77e None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:44.467663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62212ae8-9057-4803-8eb8-ed5ad81ab77e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:44.467663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62212ae8-9057-4803-8eb8-ed5ad81ab77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:44.492313 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-62212ae8-9057-4803-8eb8-ed5ad81ab77e None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:44.493599 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-62212ae8-9057-4803-8eb8-ed5ad81ab77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:44.586506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:40:44.622686 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce292e9-4dc6-4977-8510-dccfa3a80806 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:44.717727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31197-48e6-43d1-8d25-4301e50d4ab2 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:44.718512 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31197-48e6-43d1-8d25-4301e50d4ab2 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:44.719057 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31197-48e6-43d1-8d25-4301e50d4ab2 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:44.720907 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:44.727002 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557b290-358a-41ae-b062-56f880eb6928 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:44.727591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557b290-358a-41ae-b062-56f880eb6928 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:44.728108 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557b290-358a-41ae-b062-56f880eb6928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:44.745307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-d557b290-358a-41ae-b062-56f880eb6928 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:44.746953 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-d557b290-358a-41ae-b062-56f880eb6928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:44.748839 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce292e9-4dc6-4977-8510-dccfa3a80806 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:44.784167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca1486-1bea-4074-9bd0-cb7774f3a9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:44.785291 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1486-1bea-4074-9bd0-cb7774f3a9dd None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:44.785744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1486-1bea-4074-9bd0-cb7774f3a9dd None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:44.786165 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca1486-1bea-4074-9bd0-cb7774f3a9dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:44.813202 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca1486-1bea-4074-9bd0-cb7774f3a9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-4aca1486-1bea-4074-9bd0-cb7774f3a9dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:40:44.829874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:44.831142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:40:44.832276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:40:44.834456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 13/25] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:40:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:40:44.850944 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca1486-1bea-4074-9bd0-cb7774f3a9dd None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:44.854908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 13/26] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:40:44.873997 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee4d3a-d49a-4ce9-b843-4f83761fe20d None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:44.874554 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee4d3a-d49a-4ce9-b843-4f83761fe20d None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:44.875024 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee4d3a-d49a-4ce9-b843-4f83761fe20d None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:44.878665 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 14/27] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:44.883386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b4df0-5dd7-41f4-800e-ebe45f2ea703 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:44.884199 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b4df0-5dd7-41f4-800e-ebe45f2ea703 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:44.884862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b4df0-5dd7-41f4-800e-ebe45f2ea703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:44.904980 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-864b4df0-5dd7-41f4-800e-ebe45f2ea703 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:44.906613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-864b4df0-5dd7-41f4-800e-ebe45f2ea703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:45.005117 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca1486-1bea-4074-9bd0-cb7774f3a9dd None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:45.017673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 15/28] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:40:45.018979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b4df0-5dd7-41f4-800e-ebe45f2ea703 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:45.044159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:45.044849 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.045403 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.045687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.046456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:45.046775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:45.048190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:45.067619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 14/29] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:40:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:45.068721 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:45.195193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b4df0-5dd7-41f4-800e-ebe45f2ea703 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:45.237070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e30f8-f326-4576-a7a2-9695b227a549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:45.237070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e30f8-f326-4576-a7a2-9695b227a549 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.237070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e30f8-f326-4576-a7a2-9695b227a549 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.237070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e30f8-f326-4576-a7a2-9695b227a549 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.237524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e30f8-f326-4576-a7a2-9695b227a549 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:45.238056 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e30f8-f326-4576-a7a2-9695b227a549 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:45.273512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33319966-c745-4f47-bf4b-fd26b129fbb3 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:45.343406 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:45.344225 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.344517 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.344909 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.356603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:45.358285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:40:45.378782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:45.379863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:45.380853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:40:45.380853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:40:45.380853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:40:45.380853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:40:45.380853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 15/30] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:40:45.383450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:45.398124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 16/31] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:40:45 2022] POST /identity/v3/roles => generated 219 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:40:45.406301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e30f8-f326-4576-a7a2-9695b227a549 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:45.494643 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e771b2-5fcb-4291-a883-083427b8a95c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:45.546778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3dbf4-7e73-4f93-b3bb-bc0b46ba1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:45.547612 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3dbf4-7e73-4f93-b3bb-bc0b46ba1cb5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.547906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3dbf4-7e73-4f93-b3bb-bc0b46ba1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.548198 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3dbf4-7e73-4f93-b3bb-bc0b46ba1cb5 None admin] PATH_INFO: `/v3/users/d2a1baaf61604444b2c20a19956c1591` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.554887 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3dbf4-7e73-4f93-b3bb-bc0b46ba1cb5 None admin] RBAC: Authorizing `identity:get_user(user_id=d2a1baaf61604444b2c20a19956c1591)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:45.558389 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3dbf4-7e73-4f93-b3bb-bc0b46ba1cb5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:45.560132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 16/32] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:40:45 2022] GET /identity/v3/users/d2a1baaf61604444b2c20a19956c1591 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:45.564828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3dbf4-7e73-4f93-b3bb-bc0b46ba1cb5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:45.575691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e30f8-f326-4576-a7a2-9695b227a549 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:45.635073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d25f8-e089-4a30-806d-e96d002983ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:45.636096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d25f8-e089-4a30-806d-e96d002983ba None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.636549 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d25f8-e089-4a30-806d-e96d002983ba None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.636979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d25f8-e089-4a30-806d-e96d002983ba None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.638313 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d25f8-e089-4a30-806d-e96d002983ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:45.638802 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d25f8-e089-4a30-806d-e96d002983ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:45.641307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d25f8-e089-4a30-806d-e96d002983ba None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:45.669596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 17/33] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:45.671145 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187ccf-113b-4768-9810-4665ddda7a0a None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.671480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187ccf-113b-4768-9810-4665ddda7a0a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.671714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38187ccf-113b-4768-9810-4665ddda7a0a None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.681442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 18/34] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:45.682261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38187ccf-113b-4768-9810-4665ddda7a0a None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:45.692007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3dbf4-7e73-4f93-b3bb-bc0b46ba1cb5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:45.733796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d594a-c99d-4a95-a4b8-202baa4223a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:45.734623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d594a-c99d-4a95-a4b8-202baa4223a0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.734915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d594a-c99d-4a95-a4b8-202baa4223a0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.735202 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d594a-c99d-4a95-a4b8-202baa4223a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.736436 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d594a-c99d-4a95-a4b8-202baa4223a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:45.738003 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d594a-c99d-4a95-a4b8-202baa4223a0 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:45.739920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 17/35] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:45.741501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201f64-6c72-46f1-ba2e-b2e774b6cfad None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.741798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201f64-6c72-46f1-ba2e-b2e774b6cfad None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.742219 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de201f64-6c72-46f1-ba2e-b2e774b6cfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.759786 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-de201f64-6c72-46f1-ba2e-b2e774b6cfad None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:45.761069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-de201f64-6c72-46f1-ba2e-b2e774b6cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:45.818627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38187ccf-113b-4768-9810-4665ddda7a0a None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:45.844501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 18/36] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:40:45.845468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de201f64-6c72-46f1-ba2e-b2e774b6cfad None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:45.869974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e2cac-1d42-45d6-bceb-7cca2d6a2cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:45.870845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e2cac-1d42-45d6-bceb-7cca2d6a2cd1 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:45.871192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e2cac-1d42-45d6-bceb-7cca2d6a2cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:45.871535 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e2cac-1d42-45d6-bceb-7cca2d6a2cd1 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:45.873077 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e2cac-1d42-45d6-bceb-7cca2d6a2cd1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:45.874649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e2cac-1d42-45d6-bceb-7cca2d6a2cd1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:45.915757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 19/37] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:40:45 2022] POST /identity/v3/projects => generated 301 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:40:45.916962 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e2cac-1d42-45d6-bceb-7cca2d6a2cd1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:45.972559 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de201f64-6c72-46f1-ba2e-b2e774b6cfad None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:46.016276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9f6a3-695a-4ea9-ab1d-027e02591151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:46.017334 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9f6a3-695a-4ea9-ab1d-027e02591151 None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.017628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9f6a3-695a-4ea9-ab1d-027e02591151 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.018101 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9f6a3-695a-4ea9-ab1d-027e02591151 None admin] PATH_INFO: `/v3/domains/default/users/d2a1baaf61604444b2c20a19956c1591/roles/ebff922955b74c919d58613875326151` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.021692 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9f6a3-695a-4ea9-ab1d-027e02591151 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d2a1baaf61604444b2c20a19956c1591, role_id=ebff922955b74c919d58613875326151)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:46.023874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9f6a3-695a-4ea9-ab1d-027e02591151 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:46.081481 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e2cac-1d42-45d6-bceb-7cca2d6a2cd1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:46.125182 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c96079-c159-4541-8d3e-46fe96048511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:46.125892 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c96079-c159-4541-8d3e-46fe96048511 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.126184 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c96079-c159-4541-8d3e-46fe96048511 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.126469 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c96079-c159-4541-8d3e-46fe96048511 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.127375 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c96079-c159-4541-8d3e-46fe96048511 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:46.128865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c96079-c159-4541-8d3e-46fe96048511 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:46.146182 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 20/38] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:40:45 2022] POST /identity/v3/roles => generated 225 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:40:46.148232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3869384-f75f-4873-904d-0c411b532ef1 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.148581 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3869384-f75f-4873-904d-0c411b532ef1 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.148882 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3869384-f75f-4873-904d-0c411b532ef1 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.150731 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 21/39] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:46.156685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181f714-9969-4151-a73d-4fbcb323e202 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.156986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181f714-9969-4151-a73d-4fbcb323e202 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.157361 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181f714-9969-4151-a73d-4fbcb323e202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.178369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-7181f714-9969-4151-a73d-4fbcb323e202 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:46.180098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-7181f714-9969-4151-a73d-4fbcb323e202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:46.196782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 19/40] 10.222.0.13 () {58 vars in 1361 bytes} [Mon May 9 15:40:45 2022] PUT /identity/v3/domains/default/users/d2a1baaf61604444b2c20a19956c1591/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:40:46.264774 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 22/41] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:40:46.297431 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9f6a3-695a-4ea9-ab1d-027e02591151 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:46.433076 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9f6a3-695a-4ea9-ab1d-027e02591151 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:46.481571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9eeed2-5fe5-4b92-8a47-a2ee20884adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:46.481571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eeed2-5fe5-4b92-8a47-a2ee20884adc None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.481571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eeed2-5fe5-4b92-8a47-a2ee20884adc None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.481571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9eeed2-5fe5-4b92-8a47-a2ee20884adc None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.482017 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eeed2-5fe5-4b92-8a47-a2ee20884adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:46.482521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9eeed2-5fe5-4b92-8a47-a2ee20884adc None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:46.487383 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 20/42] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:46.496090 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181f714-9969-4151-a73d-4fbcb323e202 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:46.550539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0efbbd-02aa-42a8-a029-1e87cab68406 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.551097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0efbbd-02aa-42a8-a029-1e87cab68406 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.551097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0efbbd-02aa-42a8-a029-1e87cab68406 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.552655 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 21/43] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:40:46.558289 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e484b7-94df-4609-999c-f0df3754e206 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.558595 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e484b7-94df-4609-999c-f0df3754e206 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.558892 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e484b7-94df-4609-999c-f0df3754e206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.576496 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-58e484b7-94df-4609-999c-f0df3754e206 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:46.578141 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-58e484b7-94df-4609-999c-f0df3754e206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:46.611163 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181f714-9969-4151-a73d-4fbcb323e202 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:46.646146 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc7f47-ba5c-458b-b82e-4409e9a3f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:46.649227 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 22/44] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:40:46.649362 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc7f47-ba5c-458b-b82e-4409e9a3f9eb None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.649362 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc7f47-ba5c-458b-b82e-4409e9a3f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.649362 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc7f47-ba5c-458b-b82e-4409e9a3f9eb None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.649362 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc7f47-ba5c-458b-b82e-4409e9a3f9eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:46.649765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc7f47-ba5c-458b-b82e-4409e9a3f9eb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:46.670201 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e484b7-94df-4609-999c-f0df3754e206 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:46.682198 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 23/45] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:40:46 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:40:46.779878 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e484b7-94df-4609-999c-f0df3754e206 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:46.817587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedd52a-7f83-44c4-ab93-0af201b5a7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:46.818303 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedd52a-7f83-44c4-ab93-0af201b5a7e1 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:46.818601 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedd52a-7f83-44c4-ab93-0af201b5a7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:46.819468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedd52a-7f83-44c4-ab93-0af201b5a7e1 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:46.820262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedd52a-7f83-44c4-ab93-0af201b5a7e1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:46.821742 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedd52a-7f83-44c4-ab93-0af201b5a7e1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:46.839059 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 23/46] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:40:46 2022] POST /identity/v3/roles => generated 223 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:40:48.557569 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f681d99-6730-44ec-98aa-ae63f583e658 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:48.558177 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f681d99-6730-44ec-98aa-ae63f583e658 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:48.558177 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f681d99-6730-44ec-98aa-ae63f583e658 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:48.559525 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 24/47] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:48.564775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583c5f-1646-4ed2-8cdd-469213106825 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:48.565246 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583c5f-1646-4ed2-8cdd-469213106825 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:48.565373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99583c5f-1646-4ed2-8cdd-469213106825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:48.582252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-99583c5f-1646-4ed2-8cdd-469213106825 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:48.583443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-99583c5f-1646-4ed2-8cdd-469213106825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:48.647791 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 24/48] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:40:48.670373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f681d99-6730-44ec-98aa-ae63f583e658 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:48.772093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f681d99-6730-44ec-98aa-ae63f583e658 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:48.806465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2069be-0da1-4a3d-aa20-d2a9434139e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:48.807271 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2069be-0da1-4a3d-aa20-d2a9434139e5 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:48.807565 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2069be-0da1-4a3d-aa20-d2a9434139e5 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:48.807902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2069be-0da1-4a3d-aa20-d2a9434139e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:48.819972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2069be-0da1-4a3d-aa20-d2a9434139e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-5a2069be-0da1-4a3d-aa20-d2a9434139e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:40:48.828717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:48.830076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:40:48.831344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 25/49] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:40:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:40:48.838444 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99583c5f-1646-4ed2-8cdd-469213106825 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:48.933869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99583c5f-1646-4ed2-8cdd-469213106825 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:48.949420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c5c2f-cc66-4a8d-92ae-cd50c6d2d32e None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:48.949718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c5c2f-cc66-4a8d-92ae-cd50c6d2d32e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:48.950008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c5c2f-cc66-4a8d-92ae-cd50c6d2d32e None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:48.951386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 26/50] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:40:48.956347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857385f-2297-4dca-944b-2daac4302a58 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:48.956639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857385f-2297-4dca-944b-2daac4302a58 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:48.956923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857385f-2297-4dca-944b-2daac4302a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:48.967952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:48.968633 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:48.968927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:48.969239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:48.970003 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:48.970324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:48.971714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:48.973615 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-c857385f-2297-4dca-944b-2daac4302a58 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:48.974822 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-c857385f-2297-4dca-944b-2daac4302a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:48.980039 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 25/51] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:40:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:48.985600 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:49.039904 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 27/52] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:40:49.063207 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857385f-2297-4dca-944b-2daac4302a58 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:49.080765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa253cee-c9ae-4534-936d-09fb380143cb None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:49.118390 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ac374-0423-45b0-a70a-dd04015029d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:49.119312 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ac374-0423-45b0-a70a-dd04015029d5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:49.119607 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ac374-0423-45b0-a70a-dd04015029d5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:49.119895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ac374-0423-45b0-a70a-dd04015029d5 None admin] PATH_INFO: `/v3/users/d2a1baaf61604444b2c20a19956c1591` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:49.121139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ac374-0423-45b0-a70a-dd04015029d5 None admin] RBAC: Authorizing `identity:get_user(user_id=d2a1baaf61604444b2c20a19956c1591)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:49.122603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ac374-0423-45b0-a70a-dd04015029d5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:49.124263 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 26/53] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:40:48 2022] GET /identity/v3/users/d2a1baaf61604444b2c20a19956c1591 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:49.129725 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ac374-0423-45b0-a70a-dd04015029d5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:49.165590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857385f-2297-4dca-944b-2daac4302a58 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:49.198887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a3338-c5e1-4b7f-834a-0e1b7fe0006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:49.199688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a3338-c5e1-4b7f-834a-0e1b7fe0006d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:49.200045 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a3338-c5e1-4b7f-834a-0e1b7fe0006d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:49.200335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a3338-c5e1-4b7f-834a-0e1b7fe0006d None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:49.201572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a3338-c5e1-4b7f-834a-0e1b7fe0006d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:49.203047 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a3338-c5e1-4b7f-834a-0e1b7fe0006d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:49.204706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 28/54] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:49.210968 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a3338-c5e1-4b7f-834a-0e1b7fe0006d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:49.224409 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ac374-0423-45b0-a70a-dd04015029d5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:49.255856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fafbeb-bf68-4b1f-a0b4-a8171f590cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:49.257191 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fafbeb-bf68-4b1f-a0b4-a8171f590cfb None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:49.257489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fafbeb-bf68-4b1f-a0b4-a8171f590cfb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:49.257776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fafbeb-bf68-4b1f-a0b4-a8171f590cfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:49.258977 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fafbeb-bf68-4b1f-a0b4-a8171f590cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:49.260415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fafbeb-bf68-4b1f-a0b4-a8171f590cfb None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:49.262080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 27/55] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:49.267862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fafbeb-bf68-4b1f-a0b4-a8171f590cfb None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:49.304700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a3338-c5e1-4b7f-834a-0e1b7fe0006d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:49.337960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d3d5b2-26fa-4ef9-bc06-d18291358821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:49.338562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3d5b2-26fa-4ef9-bc06-d18291358821 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:49.338860 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3d5b2-26fa-4ef9-bc06-d18291358821 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:49.339151 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3d5b2-26fa-4ef9-bc06-d18291358821 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:49.339873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3d5b2-26fa-4ef9-bc06-d18291358821 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:49.341353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d3d5b2-26fa-4ef9-bc06-d18291358821 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:49.362353 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fafbeb-bf68-4b1f-a0b4-a8171f590cfb None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:49.366328 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 29/56] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 9 15:40:49 2022] POST /identity/v3/projects => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:40:49.398067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b2069-dac3-434c-9506-7bb53264d735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:49.398773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2069-dac3-434c-9506-7bb53264d735 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:49.399070 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2069-dac3-434c-9506-7bb53264d735 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:49.399360 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b2069-dac3-434c-9506-7bb53264d735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:49.400231 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b2069-dac3-434c-9506-7bb53264d735 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.domain.id=default, user.id=d2a1baaf61604444b2c20a19956c1591) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:49.400561 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b2069-dac3-434c-9506-7bb53264d735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:49.402081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b2069-dac3-434c-9506-7bb53264d735 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:49.409245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 28/57] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 9 15:40:49 2022] GET /identity/v3/role_assignments?user.id=d2a1baaf61604444b2c20a19956c1591&scope.domain.id=default&role.id=ebff922955b74c919d58613875326151 => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:40:51.615801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c22582-bbae-4595-8d02-d468245d1264 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:51.616374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c22582-bbae-4595-8d02-d468245d1264 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:51.616374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c22582-bbae-4595-8d02-d468245d1264 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:51.617811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 30/58] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:51.623137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e62d1-db54-4adf-9296-32d19615157c None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:51.623557 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e62d1-db54-4adf-9296-32d19615157c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:51.623712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e62d1-db54-4adf-9296-32d19615157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:51.640472 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-086e62d1-db54-4adf-9296-32d19615157c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:51.641708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-086e62d1-db54-4adf-9296-32d19615157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:51.704080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 29/59] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:40:51.728979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c22582-bbae-4595-8d02-d468245d1264 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:51.825382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c22582-bbae-4595-8d02-d468245d1264 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:51.856540 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8aaa3a-1527-40c1-9e09-fda4ae549a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:51.857348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aaa3a-1527-40c1-9e09-fda4ae549a3b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:51.857639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aaa3a-1527-40c1-9e09-fda4ae549a3b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:51.857920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aaa3a-1527-40c1-9e09-fda4ae549a3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:51.859099 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8aaa3a-1527-40c1-9e09-fda4ae549a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:51.860531 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8aaa3a-1527-40c1-9e09-fda4ae549a3b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:51.862141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 31/60] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:51.868056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e62d1-db54-4adf-9296-32d19615157c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:51.962243 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e62d1-db54-4adf-9296-32d19615157c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:51.994428 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37699707-4106-4288-9de4-b7e378f6679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:51.995103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699707-4106-4288-9de4-b7e378f6679e None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:51.995396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699707-4106-4288-9de4-b7e378f6679e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:51.995690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699707-4106-4288-9de4-b7e378f6679e None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:51.996405 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37699707-4106-4288-9de4-b7e378f6679e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:51.997889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37699707-4106-4288-9de4-b7e378f6679e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:52.023229 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 30/61] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:40:51 2022] POST /identity/v3/users => generated 275 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:40:55.410641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65782de9-0843-45e0-80ee-a6910ba66964 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.410641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65782de9-0843-45e0-80ee-a6910ba66964 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.410641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65782de9-0843-45e0-80ee-a6910ba66964 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.410641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 32/62] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:55.416659 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3645-3d39-42a1-be08-02dafac72aa4 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.417524 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3645-3d39-42a1-be08-02dafac72aa4 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.418018 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3645-3d39-42a1-be08-02dafac72aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.445286 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-94cd3645-3d39-42a1-be08-02dafac72aa4 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:55.446871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-94cd3645-3d39-42a1-be08-02dafac72aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:55.536196 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489450-4fb0-4390-838f-559358dc56ca None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.536823 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489450-4fb0-4390-838f-559358dc56ca None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.537933 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00489450-4fb0-4390-838f-559358dc56ca None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.539813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 33/63] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:55.544607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8871f-2e0c-4d3e-a878-40ee21f234eb None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.544900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8871f-2e0c-4d3e-a878-40ee21f234eb None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.545449 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8871f-2e0c-4d3e-a878-40ee21f234eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.552265 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 31/64] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:40:55.575516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-d2e8871f-2e0c-4d3e-a878-40ee21f234eb None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:55.577138 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-d2e8871f-2e0c-4d3e-a878-40ee21f234eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:55.612689 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd3645-3d39-42a1-be08-02dafac72aa4 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:55.694447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 34/65] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:40:55.702040 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e65433-92e9-4a9e-8174-22d11fb2e2d0 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.702346 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e65433-92e9-4a9e-8174-22d11fb2e2d0 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.702640 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e65433-92e9-4a9e-8174-22d11fb2e2d0 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.704235 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 35/66] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:55.710020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fc143-f80b-4d11-a37e-7ca926de7bed None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.710324 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fc143-f80b-4d11-a37e-7ca926de7bed None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.710616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fc143-f80b-4d11-a37e-7ca926de7bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.728237 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-611fc143-f80b-4d11-a37e-7ca926de7bed None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:55.729865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-611fc143-f80b-4d11-a37e-7ca926de7bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:55.749710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd3645-3d39-42a1-be08-02dafac72aa4 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:55.788003 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefb3d0-c79b-4bfe-bd94-ca93716e4e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:55.788826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb3d0-c79b-4bfe-bd94-ca93716e4e15 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.789413 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb3d0-c79b-4bfe-bd94-ca93716e4e15 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.789888 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb3d0-c79b-4bfe-bd94-ca93716e4e15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.802003 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefb3d0-c79b-4bfe-bd94-ca93716e4e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:40:55.804251 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 36/67] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:40:55.810229 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fc143-f80b-4d11-a37e-7ca926de7bed None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9fefb3d0-c79b-4bfe-bd94-ca93716e4e15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:40:55.810968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:55.812025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:40:55.812970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:40:55.815711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 32/68] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:40:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:40:55.822153 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefb3d0-c79b-4bfe-bd94-ca93716e4e15 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:55.923033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fc143-f80b-4d11-a37e-7ca926de7bed None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:55.943098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefb3d0-c79b-4bfe-bd94-ca93716e4e15 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:55.960542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01f494-4941-4212-a961-b8cd69a4b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:55.961661 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01f494-4941-4212-a961-b8cd69a4b4ad None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.962115 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01f494-4941-4212-a961-b8cd69a4b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.962586 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01f494-4941-4212-a961-b8cd69a4b4ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.973894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01f494-4941-4212-a961-b8cd69a4b4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-6b01f494-4941-4212-a961-b8cd69a4b4ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:40:55.981789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:55.982801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:40:55.983741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 37/69] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:40:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:40:55.984616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01f494-4941-4212-a961-b8cd69a4b4ad None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:55.986636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d2296-fc72-46a2-b524-eda61db321da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:55.987330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d2296-fc72-46a2-b524-eda61db321da None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:55.987631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d2296-fc72-46a2-b524-eda61db321da None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:55.988188 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d2296-fc72-46a2-b524-eda61db321da None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:55.988974 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d2296-fc72-46a2-b524-eda61db321da None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:55.990052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d2296-fc72-46a2-b524-eda61db321da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:55.990785 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d2296-fc72-46a2-b524-eda61db321da None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:55.999151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 33/70] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:40:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:56.004983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea646c34-cdb5-4787-869f-6cea17b6150d None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.005721 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea646c34-cdb5-4787-869f-6cea17b6150d None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.006548 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea646c34-cdb5-4787-869f-6cea17b6150d None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.007896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 34/71] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:56.008613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea646c34-cdb5-4787-869f-6cea17b6150d None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.087363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01f494-4941-4212-a961-b8cd69a4b4ad None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.108957 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea646c34-cdb5-4787-869f-6cea17b6150d None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.119231 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c6a0e-b6f3-4151-aaa7-41a01f1059a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.120042 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c6a0e-b6f3-4151-aaa7-41a01f1059a3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.120331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c6a0e-b6f3-4151-aaa7-41a01f1059a3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.120611 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c6a0e-b6f3-4151-aaa7-41a01f1059a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.130322 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c6a0e-b6f3-4151-aaa7-41a01f1059a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-e35c6a0e-b6f3-4151-aaa7-41a01f1059a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:40:56.138873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:56.139914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 38/72] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:40:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:40:56.140867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c6a0e-b6f3-4151-aaa7-41a01f1059a3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.142336 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a374fb-03cb-42ab-993d-2b8c145c35b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.143024 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a374fb-03cb-42ab-993d-2b8c145c35b7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.143323 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a374fb-03cb-42ab-993d-2b8c145c35b7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.143613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a374fb-03cb-42ab-993d-2b8c145c35b7 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.144386 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a374fb-03cb-42ab-993d-2b8c145c35b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:56.144706 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a374fb-03cb-42ab-993d-2b8c145c35b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:56.146150 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a374fb-03cb-42ab-993d-2b8c145c35b7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:56.155489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 35/73] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:56.156748 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ebdd-9f38-4c06-9b1d-cc61f0f41b59 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.157088 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ebdd-9f38-4c06-9b1d-cc61f0f41b59 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.157372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0ebdd-9f38-4c06-9b1d-cc61f0f41b59 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.158789 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 36/74] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:40:56.159843 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6a513-8cdb-47ba-8dad-3abeed88b470 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.160213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6a513-8cdb-47ba-8dad-3abeed88b470 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.160503 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6a513-8cdb-47ba-8dad-3abeed88b470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.176346 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-34c6a513-8cdb-47ba-8dad-3abeed88b470 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:56.177572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-34c6a513-8cdb-47ba-8dad-3abeed88b470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:56.238956 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c6a0e-b6f3-4151-aaa7-41a01f1059a3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.241605 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 37/75] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:40:56.249255 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6a513-8cdb-47ba-8dad-3abeed88b470 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.274027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fab39a-e34d-412a-ad8b-9b9f27e30ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.274917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fab39a-e34d-412a-ad8b-9b9f27e30ee8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.275214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fab39a-e34d-412a-ad8b-9b9f27e30ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.275592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fab39a-e34d-412a-ad8b-9b9f27e30ee8 None admin] PATH_INFO: `/v3/users/d2a1baaf61604444b2c20a19956c1591` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.277065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fab39a-e34d-412a-ad8b-9b9f27e30ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=d2a1baaf61604444b2c20a19956c1591)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:56.278553 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fab39a-e34d-412a-ad8b-9b9f27e30ee8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:56.280834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 39/76] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/users/d2a1baaf61604444b2c20a19956c1591 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:56.280968 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fab39a-e34d-412a-ad8b-9b9f27e30ee8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.342070 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6a513-8cdb-47ba-8dad-3abeed88b470 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.374052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fab39a-e34d-412a-ad8b-9b9f27e30ee8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.375431 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a475c3-b87b-4c6a-a95a-13df67084c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.376117 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a475c3-b87b-4c6a-a95a-13df67084c74 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.376414 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a475c3-b87b-4c6a-a95a-13df67084c74 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.376703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a475c3-b87b-4c6a-a95a-13df67084c74 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.377508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a475c3-b87b-4c6a-a95a-13df67084c74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:56.377827 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a475c3-b87b-4c6a-a95a-13df67084c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:56.379215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a475c3-b87b-4c6a-a95a-13df67084c74 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:56.387509 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 38/77] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:56.388711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186f474-0476-411c-8ef6-3f330f8757ac None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.389027 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186f474-0476-411c-8ef6-3f330f8757ac None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.389343 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186f474-0476-411c-8ef6-3f330f8757ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.404710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-6186f474-0476-411c-8ef6-3f330f8757ac None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:40:56.405949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-6186f474-0476-411c-8ef6-3f330f8757ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:40:56.407350 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40a8c5-3ffa-45b8-b07a-41f2d331fee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.408229 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a8c5-3ffa-45b8-b07a-41f2d331fee2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.408525 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a8c5-3ffa-45b8-b07a-41f2d331fee2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.408814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40a8c5-3ffa-45b8-b07a-41f2d331fee2 None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.423391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40a8c5-3ffa-45b8-b07a-41f2d331fee2 None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:56.424846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40a8c5-3ffa-45b8-b07a-41f2d331fee2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:56.426542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 40/78] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:56.426542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40a8c5-3ffa-45b8-b07a-41f2d331fee2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.469082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 39/79] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:40:56.470691 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186f474-0476-411c-8ef6-3f330f8757ac None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.526851 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40a8c5-3ffa-45b8-b07a-41f2d331fee2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.559006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c48d-0375-4f20-be28-469e096abe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.559804 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c48d-0375-4f20-be28-469e096abe98 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.560097 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c48d-0375-4f20-be28-469e096abe98 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.560384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89c48d-0375-4f20-be28-469e096abe98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.564018 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186f474-0476-411c-8ef6-3f330f8757ac None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.570787 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89c48d-0375-4f20-be28-469e096abe98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-cb89c48d-0375-4f20-be28-469e096abe98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:40:56.578542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:40:56.579625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 41/80] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:40:56.580591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c48d-0375-4f20-be28-469e096abe98 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.598065 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4493dd-4380-4d78-b933-4c2c6015da28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.598995 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4493dd-4380-4d78-b933-4c2c6015da28 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.599289 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4493dd-4380-4d78-b933-4c2c6015da28 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.599576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4493dd-4380-4d78-b933-4c2c6015da28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.664116 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4493dd-4380-4d78-b933-4c2c6015da28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:56.665646 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4493dd-4380-4d78-b933-4c2c6015da28 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:56.676122 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89c48d-0375-4f20-be28-469e096abe98 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.708266 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28861a-8508-4754-bb7e-65e92f7a0e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.709161 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28861a-8508-4754-bb7e-65e92f7a0e8c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.709453 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28861a-8508-4754-bb7e-65e92f7a0e8c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.709735 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28861a-8508-4754-bb7e-65e92f7a0e8c None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.710945 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28861a-8508-4754-bb7e-65e92f7a0e8c None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:56.712391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28861a-8508-4754-bb7e-65e92f7a0e8c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:56.714065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 42/81] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:56.714830 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28861a-8508-4754-bb7e-65e92f7a0e8c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.759520 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 40/82] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:40:56.760379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4493dd-4380-4d78-b933-4c2c6015da28 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.809736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28861a-8508-4754-bb7e-65e92f7a0e8c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.840909 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9dd39-3ea3-498b-ae92-f4a9fedaaaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.841613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9dd39-3ea3-498b-ae92-f4a9fedaaaae None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.841910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9dd39-3ea3-498b-ae92-f4a9fedaaaae None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.842196 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9dd39-3ea3-498b-ae92-f4a9fedaaaae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.852270 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4493dd-4380-4d78-b933-4c2c6015da28 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:56.883906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e870-588e-4040-8f4b-f1c0355b0ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:56.884694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e870-588e-4040-8f4b-f1c0355b0ca9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:56.884985 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e870-588e-4040-8f4b-f1c0355b0ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:56.885301 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e870-588e-4040-8f4b-f1c0355b0ca9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:56.886477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6e870-588e-4040-8f4b-f1c0355b0ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:56.888056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6e870-588e-4040-8f4b-f1c0355b0ca9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:56.889721 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 41/83] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:56.891651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e870-588e-4040-8f4b-f1c0355b0ca9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:56.906664 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9dd39-3ea3-498b-ae92-f4a9fedaaaae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:56.908395 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9dd39-3ea3-498b-ae92-f4a9fedaaaae None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:56.988687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e870-588e-4040-8f4b-f1c0355b0ca9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.002897 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 43/84] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:40:57.006107 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9dd39-3ea3-498b-ae92-f4a9fedaaaae None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.021339 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.022558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.022852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.023139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.023904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] RBAC: Adding query filter params (name=member) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:57.024225 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.025662 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.033690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 42/85] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:40:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:57.034333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.105981 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9dd39-3ea3-498b-ae92-f4a9fedaaaae None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.133151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdbc9a-e15a-43af-9276-6b4505a21bef None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.137783 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d20a6-813e-4c27-98af-1a34d47e4a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.138446 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d20a6-813e-4c27-98af-1a34d47e4a18 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.138747 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d20a6-813e-4c27-98af-1a34d47e4a18 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.139042 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d20a6-813e-4c27-98af-1a34d47e4a18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.166516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b2339-b1d3-4e4c-823f-07983e742094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.167389 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2339-b1d3-4e4c-823f-07983e742094 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.167683 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2339-b1d3-4e4c-823f-07983e742094 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.167971 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2339-b1d3-4e4c-823f-07983e742094 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.169217 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b2339-b1d3-4e4c-823f-07983e742094 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.170695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b2339-b1d3-4e4c-823f-07983e742094 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.172415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 43/86] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:57.173198 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b2339-b1d3-4e4c-823f-07983e742094 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.204959 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d20a6-813e-4c27-98af-1a34d47e4a18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.206455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d20a6-813e-4c27-98af-1a34d47e4a18 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.267482 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b2339-b1d3-4e4c-823f-07983e742094 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.298922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208450d8-c3ee-4939-a1ed-cc5cfde93bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.299301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 44/87] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:40:57.299597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208450d8-c3ee-4939-a1ed-cc5cfde93bcc None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.299894 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208450d8-c3ee-4939-a1ed-cc5cfde93bcc None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.300137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d20a6-813e-4c27-98af-1a34d47e4a18 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.300262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208450d8-c3ee-4939-a1ed-cc5cfde93bcc None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.300896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208450d8-c3ee-4939-a1ed-cc5cfde93bcc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.302376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208450d8-c3ee-4939-a1ed-cc5cfde93bcc None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.323639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 44/88] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:40:57 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:40:57.324456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208450d8-c3ee-4939-a1ed-cc5cfde93bcc None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.404675 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d20a6-813e-4c27-98af-1a34d47e4a18 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.430492 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208450d8-c3ee-4939-a1ed-cc5cfde93bcc None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.445458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f172-d486-4a3b-812d-189858162efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.447074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f172-d486-4a3b-812d-189858162efc None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.447586 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f172-d486-4a3b-812d-189858162efc None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.448030 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f172-d486-4a3b-812d-189858162efc None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.449529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59f172-d486-4a3b-812d-189858162efc None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.451225 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59f172-d486-4a3b-812d-189858162efc None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.453249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 45/89] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:57.454270 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f172-d486-4a3b-812d-189858162efc None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.467529 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f36e56-61f8-4578-81c9-04b5724c4dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.468446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f36e56-61f8-4578-81c9-04b5724c4dbc None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.468745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f36e56-61f8-4578-81c9-04b5724c4dbc None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.469052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f36e56-61f8-4578-81c9-04b5724c4dbc None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.470362 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f36e56-61f8-4578-81c9-04b5724c4dbc None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.471851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f36e56-61f8-4578-81c9-04b5724c4dbc None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.478807 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 45/90] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:57.478807 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f36e56-61f8-4578-81c9-04b5724c4dbc None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.567816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f172-d486-4a3b-812d-189858162efc None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.595048 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f36e56-61f8-4578-81c9-04b5724c4dbc None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.605136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.605812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.606103 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.606382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.607297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b, user.id=d2a1baaf61604444b2c20a19956c1591) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:57.607620 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.609172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.617767 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 46/91] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/role_assignments?user.id=d2a1baaf61604444b2c20a19956c1591&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=ebff922955b74c919d58613875326151 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:57.619500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.657816 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f645c-a21b-404e-9fbb-87f45c452a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.658661 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f645c-a21b-404e-9fbb-87f45c452a1c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.658948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f645c-a21b-404e-9fbb-87f45c452a1c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.659234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f645c-a21b-404e-9fbb-87f45c452a1c None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.660580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f645c-a21b-404e-9fbb-87f45c452a1c None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.662313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f645c-a21b-404e-9fbb-87f45c452a1c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.664030 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 46/92] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:57.664945 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f645c-a21b-404e-9fbb-87f45c452a1c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.763873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65631060-a786-4395-bfbe-75dc3ec98436 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.784221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f645c-a21b-404e-9fbb-87f45c452a1c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:57.819489 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.821758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.821758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.821758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:57.822301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] RBAC: Adding query filter params (role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b, user.id=2a6b5024341c429e891c0f92594e0c9b) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:57.822771 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:57.824229 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:57.832182 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 47/93] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/role_assignments?user.id=2a6b5024341c429e891c0f92594e0c9b&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:57.833104 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:57.853886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a721a-4eaa-4426-bc98-6d13ad2e36c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:57.854578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a721a-4eaa-4426-bc98-6d13ad2e36c3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:57.854871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a721a-4eaa-4426-bc98-6d13ad2e36c3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:57.855160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a721a-4eaa-4426-bc98-6d13ad2e36c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:58.015066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c627e-0325-4ea5-b2b3-cddc2f58a4c2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:58.038350 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a721a-4eaa-4426-bc98-6d13ad2e36c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:58.039878 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a721a-4eaa-4426-bc98-6d13ad2e36c3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:58.091743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:58.092728 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:58.093218 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:58.093696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:58.094826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] RBAC: Adding query filter params (role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b, user.id=2a6b5024341c429e891c0f92594e0c9b) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:58.095338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:58.105068 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:58.111865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 48/94] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/role_assignments?user.id=2a6b5024341c429e891c0f92594e0c9b&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=044a2d93445743af91a96573d39f21eb => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:58.241830 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 47/95] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:40:58.247673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:58.391504 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c264b82-3f90-41f2-b6e7-a0211e1f0636 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:58.434600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f01a98-c6bc-4bf6-b9da-8e51b759f790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:58.435918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01a98-c6bc-4bf6-b9da-8e51b759f790 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:58.436436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01a98-c6bc-4bf6-b9da-8e51b759f790 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:58.436962 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01a98-c6bc-4bf6-b9da-8e51b759f790 None admin] PATH_INFO: `/v3/projects/4f1c568272f648beab76e72830fde086` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:58.438485 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f01a98-c6bc-4bf6-b9da-8e51b759f790 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1c568272f648beab76e72830fde086)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:58.445065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f01a98-c6bc-4bf6-b9da-8e51b759f790 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:58.447984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 49/96] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:40:58 2022] GET /identity/v3/projects/4f1c568272f648beab76e72830fde086 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:40:58.453641 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a721a-4eaa-4426-bc98-6d13ad2e36c3 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:40:58.585503 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a721a-4eaa-4426-bc98-6d13ad2e36c3 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:40:58.620782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752b367-6a85-48a1-a529-60335a95c5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:40:58.621497 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752b367-6a85-48a1-a529-60335a95c5ee None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:40:58.621793 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752b367-6a85-48a1-a529-60335a95c5ee None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:40:58.622084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752b367-6a85-48a1-a529-60335a95c5ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:40:58.622947 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752b367-6a85-48a1-a529-60335a95c5ee None admin] RBAC: Adding query filter params (role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=4f1c568272f648beab76e72830fde086, user.id=2a6b5024341c429e891c0f92594e0c9b) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:40:58.623279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752b367-6a85-48a1-a529-60335a95c5ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:40:58.624757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752b367-6a85-48a1-a529-60335a95c5ee None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:40:58.631004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 48/97] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:40:58 2022] GET /identity/v3/role_assignments?user.id=2a6b5024341c429e891c0f92594e0c9b&scope.project.id=4f1c568272f648beab76e72830fde086&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:00.375325 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73098c-f77a-40b1-b35b-90e6d52b3c49 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:00.376383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73098c-f77a-40b1-b35b-90e6d52b3c49 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:00.376920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73098c-f77a-40b1-b35b-90e6d52b3c49 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:00.378669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 50/98] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:00.385464 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972308d3-850e-4ed1-9b7c-7483a039e815 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:00.385994 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972308d3-850e-4ed1-9b7c-7483a039e815 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:00.386084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972308d3-850e-4ed1-9b7c-7483a039e815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:00.403551 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-972308d3-850e-4ed1-9b7c-7483a039e815 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:00.404781 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-972308d3-850e-4ed1-9b7c-7483a039e815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:00.478104 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 49/99] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:00.524467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73098c-f77a-40b1-b35b-90e6d52b3c49 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:00.661660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73098c-f77a-40b1-b35b-90e6d52b3c49 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:00.698728 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af93e7-416b-43d1-82d4-96c2cef68f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:00.699865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af93e7-416b-43d1-82d4-96c2cef68f76 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:00.700323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af93e7-416b-43d1-82d4-96c2cef68f76 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:00.700787 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af93e7-416b-43d1-82d4-96c2cef68f76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:00.702309 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af93e7-416b-43d1-82d4-96c2cef68f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:00.704027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af93e7-416b-43d1-82d4-96c2cef68f76 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:00.706305 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 51/100] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:00.712811 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972308d3-850e-4ed1-9b7c-7483a039e815 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:00.795793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418a35c-a1a6-40ea-b67f-92a4a9909a6d None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:00.796475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418a35c-a1a6-40ea-b67f-92a4a9909a6d None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:00.796977 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418a35c-a1a6-40ea-b67f-92a4a9909a6d None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:00.798596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 52/101] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:00.816803 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750a91-68e6-4905-a2d4-5b9bdd1ce485 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:00.817841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750a91-68e6-4905-a2d4-5b9bdd1ce485 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:00.819235 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e750a91-68e6-4905-a2d4-5b9bdd1ce485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:00.838770 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972308d3-850e-4ed1-9b7c-7483a039e815 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:00.858784 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-1e750a91-68e6-4905-a2d4-5b9bdd1ce485 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:00.860990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-1e750a91-68e6-4905-a2d4-5b9bdd1ce485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:00.876312 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17ab3a-7c52-49ca-8148-07f3cd09ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:00.877002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17ab3a-7c52-49ca-8148-07f3cd09ce4f None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:00.877325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17ab3a-7c52-49ca-8148-07f3cd09ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:00.877612 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17ab3a-7c52-49ca-8148-07f3cd09ce4f None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:00.878351 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17ab3a-7c52-49ca-8148-07f3cd09ce4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:00.879988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17ab3a-7c52-49ca-8148-07f3cd09ce4f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:00.915399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 50/102] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:41:00 2022] POST /identity/v3/projects => generated 302 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:00.999190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 53/103] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:01.011038 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14239f-8207-48f0-ab7f-c33a879fc238 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.011344 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14239f-8207-48f0-ab7f-c33a879fc238 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.011737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14239f-8207-48f0-ab7f-c33a879fc238 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.013450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 51/104] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:01.019276 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442af3bf-3c89-4e54-83c1-ab8005502441 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.019584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442af3bf-3c89-4e54-83c1-ab8005502441 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.019876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442af3bf-3c89-4e54-83c1-ab8005502441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.037703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14239f-8207-48f0-ab7f-c33a879fc238 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.053264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-442af3bf-3c89-4e54-83c1-ab8005502441 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:01.057110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-442af3bf-3c89-4e54-83c1-ab8005502441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:01.195499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14239f-8207-48f0-ab7f-c33a879fc238 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:01.224181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 54/105] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:01.243700 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144bc90-7fe6-4624-92ea-424e76c15c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:01.244766 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144bc90-7fe6-4624-92ea-424e76c15c9c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.245252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144bc90-7fe6-4624-92ea-424e76c15c9c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.245709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144bc90-7fe6-4624-92ea-424e76c15c9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.252023 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442af3bf-3c89-4e54-83c1-ab8005502441 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.281978 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144bc90-7fe6-4624-92ea-424e76c15c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9144bc90-7fe6-4624-92ea-424e76c15c9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:01.290697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:01.291884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:01.292907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:01.295908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 52/106] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:01.297482 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caffbe0-d996-4d75-a4ff-2b8605100d7e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.297991 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caffbe0-d996-4d75-a4ff-2b8605100d7e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.298470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caffbe0-d996-4d75-a4ff-2b8605100d7e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.308835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 53/107] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:01.310060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caffbe0-d996-4d75-a4ff-2b8605100d7e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.374957 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442af3bf-3c89-4e54-83c1-ab8005502441 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:01.409516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38532d7-7a83-48e2-83d3-256144fb7bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:01.411091 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38532d7-7a83-48e2-83d3-256144fb7bfb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.411567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38532d7-7a83-48e2-83d3-256144fb7bfb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.412057 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38532d7-7a83-48e2-83d3-256144fb7bfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.423478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38532d7-7a83-48e2-83d3-256144fb7bfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-f38532d7-7a83-48e2-83d3-256144fb7bfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:01.432386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:01.433506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:01.434429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:01.435139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caffbe0-d996-4d75-a4ff-2b8605100d7e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:01.437421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 55/108] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:01.439015 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2072c0-cefb-46b8-9c2c-ae4673ede0ab None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.439552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2072c0-cefb-46b8-9c2c-ae4673ede0ab None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.440050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2072c0-cefb-46b8-9c2c-ae4673ede0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.457688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-bf2072c0-cefb-46b8-9c2c-ae4673ede0ab None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:01.459304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-bf2072c0-cefb-46b8-9c2c-ae4673ede0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:01.470366 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:01.471399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.471527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.471705 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.472554 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:01.472854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:01.475190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:01.484861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 54/109] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:01.485848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.533467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 56/110] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:01.534611 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2072c0-cefb-46b8-9c2c-ae4673ede0ab None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.594949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e36f0-eece-40f3-b88f-d2030ed3def4 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:01.630066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:01.631494 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.631494 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.631494 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.632516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] RBAC: Adding query filter params (name=member) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:01.632925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:01.634443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:01.643110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 55/111] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:01.644180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.663667 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2072c0-cefb-46b8-9c2c-ae4673ede0ab None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:01.698972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6e482-d0b6-414f-980c-9db1cd053932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:01.700218 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6e482-d0b6-414f-980c-9db1cd053932 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.700771 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6e482-d0b6-414f-980c-9db1cd053932 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.701289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6e482-d0b6-414f-980c-9db1cd053932 None admin] PATH_INFO: `/v3/users/d2a1baaf61604444b2c20a19956c1591` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.702725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6e482-d0b6-414f-980c-9db1cd053932 None admin] RBAC: Authorizing `identity:get_user(user_id=d2a1baaf61604444b2c20a19956c1591)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:01.704414 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6e482-d0b6-414f-980c-9db1cd053932 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:01.706391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 57/112] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/users/d2a1baaf61604444b2c20a19956c1591 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:01.707421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6e482-d0b6-414f-980c-9db1cd053932 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.746848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac04989-4170-4b1d-80c6-790929d3c2ed None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:01.781208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8244795-aae2-4c89-b97e-62d62f61690f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:01.782081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8244795-aae2-4c89-b97e-62d62f61690f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.782375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8244795-aae2-4c89-b97e-62d62f61690f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.782657 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8244795-aae2-4c89-b97e-62d62f61690f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.793958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8244795-aae2-4c89-b97e-62d62f61690f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-c8244795-aae2-4c89-b97e-62d62f61690f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:01.801870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:01.803013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:01.803966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 56/113] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:01.804778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8244795-aae2-4c89-b97e-62d62f61690f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.821095 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6e482-d0b6-414f-980c-9db1cd053932 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:01.862395 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fbb90-f8cc-438f-a4c3-5b5f969ee482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:01.862789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbb90-f8cc-438f-a4c3-5b5f969ee482 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:01.865135 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbb90-f8cc-438f-a4c3-5b5f969ee482 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:01.865135 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3fbb90-f8cc-438f-a4c3-5b5f969ee482 None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:01.866295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fbb90-f8cc-438f-a4c3-5b5f969ee482 None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:01.867809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3fbb90-f8cc-438f-a4c3-5b5f969ee482 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:01.872840 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 58/114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:01.875461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fbb90-f8cc-438f-a4c3-5b5f969ee482 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:01.963656 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8244795-aae2-4c89-b97e-62d62f61690f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.017937 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46945588-232f-498e-9065-37336abde1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.019143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945588-232f-498e-9065-37336abde1a5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.020583 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945588-232f-498e-9065-37336abde1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.020885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945588-232f-498e-9065-37336abde1a5 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.023414 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46945588-232f-498e-9065-37336abde1a5 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:02.025134 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46945588-232f-498e-9065-37336abde1a5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:02.027197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 57/115] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:02.028174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46945588-232f-498e-9065-37336abde1a5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.082133 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3fbb90-f8cc-438f-a4c3-5b5f969ee482 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.123361 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.124085 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.124378 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.124717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.125543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:02.125912 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:02.127384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:02.141442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 59/116] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:02.142339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.149765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46945588-232f-498e-9065-37336abde1a5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.192877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c8017-c4d7-4d7a-bcc9-54018245316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.193834 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c8017-c4d7-4d7a-bcc9-54018245316e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.194128 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c8017-c4d7-4d7a-bcc9-54018245316e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.194421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c8017-c4d7-4d7a-bcc9-54018245316e None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.195635 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c8017-c4d7-4d7a-bcc9-54018245316e None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:02.197156 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c8017-c4d7-4d7a-bcc9-54018245316e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:02.199084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 58/117] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:02 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:02.200097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21a14b-9f5e-43a4-947d-34e7ac978ea0 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.200401 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21a14b-9f5e-43a4-947d-34e7ac978ea0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.200694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21a14b-9f5e-43a4-947d-34e7ac978ea0 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.202179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 59/118] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:02.202705 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21a14b-9f5e-43a4-947d-34e7ac978ea0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.250099 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6bc0e-0f4c-43ce-b660-b3c9f1ec2df1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.283910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75573736-58c0-4710-a94b-04e053baf884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.284739 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75573736-58c0-4710-a94b-04e053baf884 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.285030 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75573736-58c0-4710-a94b-04e053baf884 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.285343 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75573736-58c0-4710-a94b-04e053baf884 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/users/d2a1baaf61604444b2c20a19956c1591/roles/ebff922955b74c919d58613875326151` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.287974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75573736-58c0-4710-a94b-04e053baf884 None admin] RBAC: Authorizing `identity:create_grant(project_id=38e69abc282f4bf1aa231ecc15fc213b, user_id=d2a1baaf61604444b2c20a19956c1591, role_id=ebff922955b74c919d58613875326151)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:02.289491 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75573736-58c0-4710-a94b-04e053baf884 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:02.303935 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 60/119] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:02 2022] PUT /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/users/d2a1baaf61604444b2c20a19956c1591/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:02.306783 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e68e04-2f2c-4ec0-ae6c-615a201af1d1 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.307094 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e68e04-2f2c-4ec0-ae6c-615a201af1d1 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.307392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e68e04-2f2c-4ec0-ae6c-615a201af1d1 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.309101 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 61/120] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:02.314092 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e68e04-2f2c-4ec0-ae6c-615a201af1d1 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.314306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21a14b-9f5e-43a4-947d-34e7ac978ea0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.356450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2a4d0-c952-4bd2-8dd5-0db8ba5a5087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.357931 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a4d0-c952-4bd2-8dd5-0db8ba5a5087 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.358509 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a4d0-c952-4bd2-8dd5-0db8ba5a5087 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.359081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a4d0-c952-4bd2-8dd5-0db8ba5a5087 None admin] PATH_INFO: `/v3/users/c187e47f86364d0f9b952b57e6548b83` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.375879 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2a4d0-c952-4bd2-8dd5-0db8ba5a5087 None admin] RBAC: Authorizing `identity:get_user(user_id=c187e47f86364d0f9b952b57e6548b83)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:02.377871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2a4d0-c952-4bd2-8dd5-0db8ba5a5087 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:02.380032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 60/121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:02 2022] GET /identity/v3/users/c187e47f86364d0f9b952b57e6548b83 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:02.381880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f733d34-31eb-4492-ab2f-9200be7481fd None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.382455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f733d34-31eb-4492-ab2f-9200be7481fd None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.383046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f733d34-31eb-4492-ab2f-9200be7481fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.402978 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-3f733d34-31eb-4492-ab2f-9200be7481fd None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:02.405650 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-3f733d34-31eb-4492-ab2f-9200be7481fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:02.434654 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e68e04-2f2c-4ec0-ae6c-615a201af1d1 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.472656 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e8cd5-3199-4ca0-8e7e-f5087172601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.474240 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e8cd5-3199-4ca0-8e7e-f5087172601f None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.474613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e8cd5-3199-4ca0-8e7e-f5087172601f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.475395 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e8cd5-3199-4ca0-8e7e-f5087172601f None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/users/2a6b5024341c429e891c0f92594e0c9b/roles/22b9b0cd5e41452380f6aff61caa9f91` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.478327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e8cd5-3199-4ca0-8e7e-f5087172601f None admin] RBAC: Authorizing `identity:create_grant(project_id=38e69abc282f4bf1aa231ecc15fc213b, user_id=2a6b5024341c429e891c0f92594e0c9b, role_id=22b9b0cd5e41452380f6aff61caa9f91)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:02.480326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e8cd5-3199-4ca0-8e7e-f5087172601f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:02.487339 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 61/122] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:02.490369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08c0ef-2d82-4adb-a02a-d323ec50d1f4 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.490982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08c0ef-2d82-4adb-a02a-d323ec50d1f4 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.491500 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08c0ef-2d82-4adb-a02a-d323ec50d1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.499224 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 62/123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:02 2022] PUT /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/users/2a6b5024341c429e891c0f92594e0c9b/roles/22b9b0cd5e41452380f6aff61caa9f91 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:02.499224 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e8cd5-3199-4ca0-8e7e-f5087172601f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.518306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-6a08c0ef-2d82-4adb-a02a-d323ec50d1f4 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:02.519920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-6a08c0ef-2d82-4adb-a02a-d323ec50d1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:02.605754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 62/124] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:02.606429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08c0ef-2d82-4adb-a02a-d323ec50d1f4 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.615439 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e8cd5-3199-4ca0-8e7e-f5087172601f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.672596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1faed6-304d-4e87-93f5-44519d75bfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.673308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1faed6-304d-4e87-93f5-44519d75bfdf None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.673609 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1faed6-304d-4e87-93f5-44519d75bfdf None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.673901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1faed6-304d-4e87-93f5-44519d75bfdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.715285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08c0ef-2d82-4adb-a02a-d323ec50d1f4 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.746964 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1faed6-304d-4e87-93f5-44519d75bfdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:02.748489 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1faed6-304d-4e87-93f5-44519d75bfdf None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:02.751581 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64f0c9-a533-4dd1-b98e-beb686282118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.752405 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64f0c9-a533-4dd1-b98e-beb686282118 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.752754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64f0c9-a533-4dd1-b98e-beb686282118 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.753054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64f0c9-a533-4dd1-b98e-beb686282118 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.764369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64f0c9-a533-4dd1-b98e-beb686282118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-ec64f0c9-a533-4dd1-b98e-beb686282118 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:02.773208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:02.774324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 63/125] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:02.775688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64f0c9-a533-4dd1-b98e-beb686282118 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.853393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 63/126] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:02.854399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1faed6-304d-4e87-93f5-44519d75bfdf None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.887235 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64f0c9-a533-4dd1-b98e-beb686282118 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:02.924413 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d116d-3537-4246-856f-f08ff78b5b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:02.925306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d116d-3537-4246-856f-f08ff78b5b07 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:02.925600 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d116d-3537-4246-856f-f08ff78b5b07 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:02.925893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d116d-3537-4246-856f-f08ff78b5b07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:02.936549 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d116d-3537-4246-856f-f08ff78b5b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-a15d116d-3537-4246-856f-f08ff78b5b07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:02.946445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:02.948801 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 64/127] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:02.951068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d116d-3537-4246-856f-f08ff78b5b07 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:02.970099 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1faed6-304d-4e87-93f5-44519d75bfdf None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.005582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.006892 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.007195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.007486 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.008261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:03.008582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.010025 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.018420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 64/128] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:03.019243 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:03.056847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d116d-3537-4246-856f-f08ff78b5b07 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.092080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2c6c-fcaa-4acd-bab4-be52ebd18cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.092974 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2c6c-fcaa-4acd-bab4-be52ebd18cb2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.093296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2c6c-fcaa-4acd-bab4-be52ebd18cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.093583 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec2c6c-fcaa-4acd-bab4-be52ebd18cb2 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.094875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2c6c-fcaa-4acd-bab4-be52ebd18cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.096854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec2c6c-fcaa-4acd-bab4-be52ebd18cb2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.100123 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 65/129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:02 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:03.101118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2c6c-fcaa-4acd-bab4-be52ebd18cb2 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:03.136478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb4f8f-1f17-4c0a-b75c-7ca6aac8cf3f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.172990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.173712 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.174006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.174291 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.175066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:03.175386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.176780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.188729 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 65/130] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:03.189649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:03.218998 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec2c6c-fcaa-4acd-bab4-be52ebd18cb2 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.293754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6483e3e-bddd-417e-9fad-4695619f838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.295052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6483e3e-bddd-417e-9fad-4695619f838a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.295563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6483e3e-bddd-417e-9fad-4695619f838a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.296077 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6483e3e-bddd-417e-9fad-4695619f838a None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.297766 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6483e3e-bddd-417e-9fad-4695619f838a None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.299459 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6483e3e-bddd-417e-9fad-4695619f838a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.299776 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ead0-d6d3-42e6-b446-529b9b66f641 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.301724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 66/131] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:03 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:03.306615 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6483e3e-bddd-417e-9fad-4695619f838a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:03.334741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.335416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.336088 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.336088 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.336831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] RBAC: Adding query filter params (role.id=9178c520b46b41e48fc1659fe0129123, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b, user.id=c187e47f86364d0f9b952b57e6548b83) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:03.337187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.338636 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.344504 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 66/132] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:03 2022] GET /identity/v3/role_assignments?user.id=c187e47f86364d0f9b952b57e6548b83&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=9178c520b46b41e48fc1659fe0129123 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:03.345145 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:03.422190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6483e3e-bddd-417e-9fad-4695619f838a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.451776 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee914c0-e79a-44c7-9429-e993ec061576 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.463370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e4dfc-7d7d-4470-b617-d76553d65224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.464656 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4dfc-7d7d-4470-b617-d76553d65224 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.465217 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4dfc-7d7d-4470-b617-d76553d65224 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.465894 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e4dfc-7d7d-4470-b617-d76553d65224 None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.467475 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e4dfc-7d7d-4470-b617-d76553d65224 None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.469322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e4dfc-7d7d-4470-b617-d76553d65224 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.471402 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 67/133] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:03 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:03.482151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e4dfc-7d7d-4470-b617-d76553d65224 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:03.490056 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38ade6-2c12-48da-8068-6d361e4140dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.490996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38ade6-2c12-48da-8068-6d361e4140dd None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.491294 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38ade6-2c12-48da-8068-6d361e4140dd None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.491584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38ade6-2c12-48da-8068-6d361e4140dd None admin] PATH_INFO: `/v3/projects/4f1c568272f648beab76e72830fde086` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.492883 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38ade6-2c12-48da-8068-6d361e4140dd None admin] RBAC: Authorizing `identity:get_project(project_id=4f1c568272f648beab76e72830fde086)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.494525 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38ade6-2c12-48da-8068-6d361e4140dd None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.496539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 67/134] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:03 2022] GET /identity/v3/projects/4f1c568272f648beab76e72830fde086 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:03.502139 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38ade6-2c12-48da-8068-6d361e4140dd None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:03.609618 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38ade6-2c12-48da-8068-6d361e4140dd None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.633263 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e4dfc-7d7d-4470-b617-d76553d65224 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.644456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f7f4-0327-41af-9a52-610da9dcbbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.645400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f7f4-0327-41af-9a52-610da9dcbbe8 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.645697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f7f4-0327-41af-9a52-610da9dcbbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.645986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f7f4-0327-41af-9a52-610da9dcbbe8 None admin] PATH_INFO: `/v3/projects/4f1c568272f648beab76e72830fde086/users/2a6b5024341c429e891c0f92594e0c9b/roles/22b9b0cd5e41452380f6aff61caa9f91` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.648651 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f7f4-0327-41af-9a52-610da9dcbbe8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f1c568272f648beab76e72830fde086, user_id=2a6b5024341c429e891c0f92594e0c9b, role_id=22b9b0cd5e41452380f6aff61caa9f91)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.650157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f7f4-0327-41af-9a52-610da9dcbbe8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.669865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 68/135] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:03 2022] PUT /identity/v3/projects/4f1c568272f648beab76e72830fde086/users/2a6b5024341c429e891c0f92594e0c9b/roles/22b9b0cd5e41452380f6aff61caa9f91 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:03.692508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb27fd-ac5d-439a-8125-80b52a036485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.694019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb27fd-ac5d-439a-8125-80b52a036485 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.694317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb27fd-ac5d-439a-8125-80b52a036485 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.694606 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb27fd-ac5d-439a-8125-80b52a036485 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.695888 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb27fd-ac5d-439a-8125-80b52a036485 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.697381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb27fd-ac5d-439a-8125-80b52a036485 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.699175 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 68/136] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:03 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:03.705606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f7f4-0327-41af-9a52-610da9dcbbe8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:03.837951 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f7f4-0327-41af-9a52-610da9dcbbe8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:03.876060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8797ddc-6fca-4b41-a979-f5555a64ca68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:03.876987 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8797ddc-6fca-4b41-a979-f5555a64ca68 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:03.877317 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8797ddc-6fca-4b41-a979-f5555a64ca68 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:03.877610 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8797ddc-6fca-4b41-a979-f5555a64ca68 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/users/2a6b5024341c429e891c0f92594e0c9b/roles/044a2d93445743af91a96573d39f21eb` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:03.880038 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8797ddc-6fca-4b41-a979-f5555a64ca68 None admin] RBAC: Authorizing `identity:create_grant(project_id=38e69abc282f4bf1aa231ecc15fc213b, user_id=2a6b5024341c429e891c0f92594e0c9b, role_id=044a2d93445743af91a96573d39f21eb)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:03.882268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8797ddc-6fca-4b41-a979-f5555a64ca68 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:03.909603 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 69/137] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:03 2022] PUT /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/users/2a6b5024341c429e891c0f92594e0c9b/roles/044a2d93445743af91a96573d39f21eb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:04.381187 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335e74-4dfa-4706-b170-7db88901883a None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:04.381592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335e74-4dfa-4706-b170-7db88901883a None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:04.381800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335e74-4dfa-4706-b170-7db88901883a None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:04.383497 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 69/138] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:04.394689 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54674a-f38a-434a-8e1d-eaf9da024284 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:04.394996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54674a-f38a-434a-8e1d-eaf9da024284 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:04.395295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54674a-f38a-434a-8e1d-eaf9da024284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:04.419520 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-3e54674a-f38a-434a-8e1d-eaf9da024284 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:04.420745 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-3e54674a-f38a-434a-8e1d-eaf9da024284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:04.502246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 70/139] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:04.551814 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5335e74-4dfa-4706-b170-7db88901883a None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:04.674222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5335e74-4dfa-4706-b170-7db88901883a None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:04.708790 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054add1-7661-43c1-bda1-9395ef0b7a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:04.710444 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054add1-7661-43c1-bda1-9395ef0b7a6b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:04.710925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054add1-7661-43c1-bda1-9395ef0b7a6b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:04.711366 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054add1-7661-43c1-bda1-9395ef0b7a6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:04.712773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054add1-7661-43c1-bda1-9395ef0b7a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:04.714497 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054add1-7661-43c1-bda1-9395ef0b7a6b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:04.716327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 70/140] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:04.722098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54674a-f38a-434a-8e1d-eaf9da024284 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:04.857986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54674a-f38a-434a-8e1d-eaf9da024284 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:04.901207 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e146e23-61f3-4b58-82cd-1f3f6e143dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:04.902284 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e146e23-61f3-4b58-82cd-1f3f6e143dd7 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:04.902751 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e146e23-61f3-4b58-82cd-1f3f6e143dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:04.903212 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e146e23-61f3-4b58-82cd-1f3f6e143dd7 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:04.904117 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e146e23-61f3-4b58-82cd-1f3f6e143dd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:04.905820 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e146e23-61f3-4b58-82cd-1f3f6e143dd7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:04.951926 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 71/141] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:04 2022] POST /identity/v3/users => generated 283 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:05.734787 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3235b1-8ff7-4126-b470-58b7fea0bb6e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:05.735348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3235b1-8ff7-4126-b470-58b7fea0bb6e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:05.735348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3235b1-8ff7-4126-b470-58b7fea0bb6e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:05.736748 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 71/142] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:05.743862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31d74b-c436-4edf-ba98-3365e5e069df None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:05.744294 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31d74b-c436-4edf-ba98-3365e5e069df None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:05.744516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31d74b-c436-4edf-ba98-3365e5e069df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:05.761973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-4f31d74b-c436-4edf-ba98-3365e5e069df None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:05.763176 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-4f31d74b-c436-4edf-ba98-3365e5e069df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:05.842582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 72/143] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:05.881209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3235b1-8ff7-4126-b470-58b7fea0bb6e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:05.999561 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3235b1-8ff7-4126-b470-58b7fea0bb6e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:06.034655 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076a6ef-d9d8-48e9-aeb4-4968860e9653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:06.035482 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076a6ef-d9d8-48e9-aeb4-4968860e9653 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.035776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076a6ef-d9d8-48e9-aeb4-4968860e9653 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.036056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076a6ef-d9d8-48e9-aeb4-4968860e9653 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.047887 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076a6ef-d9d8-48e9-aeb4-4968860e9653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-0076a6ef-d9d8-48e9-aeb4-4968860e9653 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:06.056001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:06.057091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:06.058028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 72/144] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:06.063565 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31d74b-c436-4edf-ba98-3365e5e069df None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:06.177266 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31d74b-c436-4edf-ba98-3365e5e069df None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:06.212211 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea0f48-62e9-4709-b97e-219440aee62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:06.212906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0f48-62e9-4709-b97e-219440aee62e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.213231 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0f48-62e9-4709-b97e-219440aee62e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.213520 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea0f48-62e9-4709-b97e-219440aee62e None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.214293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea0f48-62e9-4709-b97e-219440aee62e None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:06.214613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea0f48-62e9-4709-b97e-219440aee62e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:06.216006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea0f48-62e9-4709-b97e-219440aee62e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:06.227174 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 73/145] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:06.232721 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076a6ef-d9d8-48e9-aeb4-4968860e9653 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:06.246713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c495d0e-7ea1-488a-ac96-f927ada1936c None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.247049 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c495d0e-7ea1-488a-ac96-f927ada1936c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.247295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c495d0e-7ea1-488a-ac96-f927ada1936c None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.249008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 74/146] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:06.262817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75bf06-d6be-4a58-ad5c-1a0468c4707a None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.263119 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75bf06-d6be-4a58-ad5c-1a0468c4707a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.263322 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75bf06-d6be-4a58-ad5c-1a0468c4707a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.289870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-0b75bf06-d6be-4a58-ad5c-1a0468c4707a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:06.291213 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-0b75bf06-d6be-4a58-ad5c-1a0468c4707a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:06.353787 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076a6ef-d9d8-48e9-aeb4-4968860e9653 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:06.394815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272bf9-4f78-488b-80ab-04c8318b44ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:06.397752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272bf9-4f78-488b-80ab-04c8318b44ba None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.401400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272bf9-4f78-488b-80ab-04c8318b44ba None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.401951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272bf9-4f78-488b-80ab-04c8318b44ba None admin] PATH_INFO: `/v3/users/d2a1baaf61604444b2c20a19956c1591` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.403524 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272bf9-4f78-488b-80ab-04c8318b44ba None admin] RBAC: Authorizing `identity:get_user(user_id=d2a1baaf61604444b2c20a19956c1591)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:06.406465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272bf9-4f78-488b-80ab-04c8318b44ba None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:06.412318 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 73/147] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:06 2022] GET /identity/v3/users/d2a1baaf61604444b2c20a19956c1591 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:06.415543 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272bf9-4f78-488b-80ab-04c8318b44ba None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:06.434710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 75/148] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:06.537993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75bf06-d6be-4a58-ad5c-1a0468c4707a None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:06.548608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272bf9-4f78-488b-80ab-04c8318b44ba None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:06.592892 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89164ade-f2a3-4307-86be-48d1cfb00b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:06.594338 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164ade-f2a3-4307-86be-48d1cfb00b5c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.594850 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164ade-f2a3-4307-86be-48d1cfb00b5c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.595361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89164ade-f2a3-4307-86be-48d1cfb00b5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.686667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89164ade-f2a3-4307-86be-48d1cfb00b5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:06.688686 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89164ade-f2a3-4307-86be-48d1cfb00b5c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:06.798824 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75bf06-d6be-4a58-ad5c-1a0468c4707a None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:06.803855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 74/149] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:06.809707 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89164ade-f2a3-4307-86be-48d1cfb00b5c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:06.845903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c801a1-31c5-4609-a6c9-e5301fe3a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:06.846633 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c801a1-31c5-4609-a6c9-e5301fe3a6f1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.846936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c801a1-31c5-4609-a6c9-e5301fe3a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.847223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c801a1-31c5-4609-a6c9-e5301fe3a6f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.862525 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c801a1-31c5-4609-a6c9-e5301fe3a6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-d8c801a1-31c5-4609-a6c9-e5301fe3a6f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:06.874682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:06.875806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:06.876738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 76/150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:06.877540 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e90d1-f98b-4bb3-899b-ab8407c0e1f8 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.878137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e90d1-f98b-4bb3-899b-ab8407c0e1f8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.878137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e90d1-f98b-4bb3-899b-ab8407c0e1f8 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.879380 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 77/151] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:06.880487 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54884c-282e-40d8-a086-893e907080d5 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.880827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54884c-282e-40d8-a086-893e907080d5 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.881141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54884c-282e-40d8-a086-893e907080d5 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.882390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 78/152] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:06.888014 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54884c-282e-40d8-a086-893e907080d5 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:06.939612 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89164ade-f2a3-4307-86be-48d1cfb00b5c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:06.983208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a409ac-36e7-4d54-b7e1-ff7bca33b9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:06.984388 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a409ac-36e7-4d54-b7e1-ff7bca33b9cb None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.984701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a409ac-36e7-4d54-b7e1-ff7bca33b9cb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.984995 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a409ac-36e7-4d54-b7e1-ff7bca33b9cb None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:06.986808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a409ac-36e7-4d54-b7e1-ff7bca33b9cb None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:06.988603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a409ac-36e7-4d54-b7e1-ff7bca33b9cb None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:06.990890 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 75/153] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:06 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:06.992181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417dbbe-ad32-426e-acc1-1c8c17fb62e8 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:06.992480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417dbbe-ad32-426e-acc1-1c8c17fb62e8 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:06.992773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417dbbe-ad32-426e-acc1-1c8c17fb62e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.008424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54884c-282e-40d8-a086-893e907080d5 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:07.015024 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-9417dbbe-ad32-426e-acc1-1c8c17fb62e8 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:07.016685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-9417dbbe-ad32-426e-acc1-1c8c17fb62e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:07.045162 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d441a74-f760-4872-ace8-d7dde2272fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:07.045859 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d441a74-f760-4872-ace8-d7dde2272fc1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:07.046159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d441a74-f760-4872-ace8-d7dde2272fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:07.046449 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d441a74-f760-4872-ace8-d7dde2272fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.047225 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d441a74-f760-4872-ace8-d7dde2272fc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:07.047554 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d441a74-f760-4872-ace8-d7dde2272fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:07.049099 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d441a74-f760-4872-ace8-d7dde2272fc1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:07.058377 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 79/154] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:07.059731 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45fbb3-3dd5-41aa-94cf-0d98bafbe0fc None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:07.060037 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45fbb3-3dd5-41aa-94cf-0d98bafbe0fc None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:07.060332 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45fbb3-3dd5-41aa-94cf-0d98bafbe0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.081965 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-ad45fbb3-3dd5-41aa-94cf-0d98bafbe0fc None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:07.083194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-ad45fbb3-3dd5-41aa-94cf-0d98bafbe0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:07.097810 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 76/155] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:07.099361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417dbbe-ad32-426e-acc1-1c8c17fb62e8 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:07.217364 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 80/156] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:07.218277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45fbb3-3dd5-41aa-94cf-0d98bafbe0fc None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:07.228037 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417dbbe-ad32-426e-acc1-1c8c17fb62e8 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:07.284608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:07.285680 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:07.285980 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:07.286272 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.287170 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b, user.id=d2a1baaf61604444b2c20a19956c1591) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:07.287497 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:07.288973 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:07.296134 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 77/157] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:07 2022] GET /identity/v3/role_assignments?user.id=d2a1baaf61604444b2c20a19956c1591&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=ebff922955b74c919d58613875326151 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:07.296978 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:07.410484 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911b83-a849-459d-b9fe-5f6e951d7786 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:07.422593 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45fbb3-3dd5-41aa-94cf-0d98bafbe0fc None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:07.453066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd79cc5-00a4-4abb-88ea-84d226fed1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:07.453066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd79cc5-00a4-4abb-88ea-84d226fed1e4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:07.453066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd79cc5-00a4-4abb-88ea-84d226fed1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:07.453066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd79cc5-00a4-4abb-88ea-84d226fed1e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.464566 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd79cc5-00a4-4abb-88ea-84d226fed1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-6dd79cc5-00a4-4abb-88ea-84d226fed1e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:07.472526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:07.473951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:07.474918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 78/158] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:07.475720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd79cc5-00a4-4abb-88ea-84d226fed1e4 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:07.484167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdfb27-4eb8-4272-b0e5-f0d93fee4ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:07.485091 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdfb27-4eb8-4272-b0e5-f0d93fee4ce3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:07.485392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdfb27-4eb8-4272-b0e5-f0d93fee4ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:07.485675 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdfb27-4eb8-4272-b0e5-f0d93fee4ce3 None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.486921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdfb27-4eb8-4272-b0e5-f0d93fee4ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:07.488397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdfb27-4eb8-4272-b0e5-f0d93fee4ce3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:07.490770 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 81/159] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:07 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:07.491410 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdfb27-4eb8-4272-b0e5-f0d93fee4ce3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:07.586702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd79cc5-00a4-4abb-88ea-84d226fed1e4 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:07.629938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c651e5-4cf2-4c04-9bde-09100490a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:07.631723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c651e5-4cf2-4c04-9bde-09100490a619 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:07.632230 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c651e5-4cf2-4c04-9bde-09100490a619 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:07.632746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c651e5-4cf2-4c04-9bde-09100490a619 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.643834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdfb27-4eb8-4272-b0e5-f0d93fee4ce3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:07.658215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c651e5-4cf2-4c04-9bde-09100490a619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-e7c651e5-4cf2-4c04-9bde-09100490a619 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:07.666542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:07.667636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:07.668573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:07.673637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 79/160] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:07.674612 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c651e5-4cf2-4c04-9bde-09100490a619 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:07.702737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:07.703452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:07.703821 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:07.704113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.704810 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:07.705155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:07.706575 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:07.720515 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 82/161] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:07.721465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:07.888257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c651e5-4cf2-4c04-9bde-09100490a619 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:07.930322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670251-9b03-4d0e-9460-19e9f22d5e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:07.931375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670251-9b03-4d0e-9460-19e9f22d5e38 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:07.931922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670251-9b03-4d0e-9460-19e9f22d5e38 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:07.932429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670251-9b03-4d0e-9460-19e9f22d5e38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:07.935057 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73807d-c473-4450-aeac-fe71321d794a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:08.006881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670251-9b03-4d0e-9460-19e9f22d5e38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:08.008937 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670251-9b03-4d0e-9460-19e9f22d5e38 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:08.020165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:08.021277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.021823 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.022322 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:08.023359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] RBAC: Adding query filter params (name=reader) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:08.023945 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:08.025639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:08.043322 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 83/162] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:08.044501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:08.119228 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 80/163] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:08.120474 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670251-9b03-4d0e-9460-19e9f22d5e38 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:08.224842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670251-9b03-4d0e-9460-19e9f22d5e38 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:08.260378 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adce1c-432b-4fd4-82ca-fdfa04f89406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:08.261675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adce1c-432b-4fd4-82ca-fdfa04f89406 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.262185 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adce1c-432b-4fd4-82ca-fdfa04f89406 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.262684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adce1c-432b-4fd4-82ca-fdfa04f89406 None admin] PATH_INFO: `/v3/users/c187e47f86364d0f9b952b57e6548b83` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:08.264161 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adce1c-432b-4fd4-82ca-fdfa04f89406 None admin] RBAC: Authorizing `identity:get_user(user_id=c187e47f86364d0f9b952b57e6548b83)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:08.265852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adce1c-432b-4fd4-82ca-fdfa04f89406 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:08.267752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 81/164] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:08 2022] GET /identity/v3/users/c187e47f86364d0f9b952b57e6548b83 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:08.269190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434f332-3e28-4976-9973-971b09276c38 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.269699 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434f332-3e28-4976-9973-971b09276c38 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.270213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434f332-3e28-4976-9973-971b09276c38 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:08.276288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 82/165] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:08.276873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d68d72-ca60-476c-bd2f-cc8800258784 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:08.277508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434f332-3e28-4976-9973-971b09276c38 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:08.352299 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6291fc-23d1-42ca-84fd-cbcbc64e12b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:08.353270 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6291fc-23d1-42ca-84fd-cbcbc64e12b9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.353560 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6291fc-23d1-42ca-84fd-cbcbc64e12b9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.353899 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6291fc-23d1-42ca-84fd-cbcbc64e12b9 None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:08.355166 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6291fc-23d1-42ca-84fd-cbcbc64e12b9 None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:08.356712 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6291fc-23d1-42ca-84fd-cbcbc64e12b9 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:08.358421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 84/166] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:08 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:08.359248 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6291fc-23d1-42ca-84fd-cbcbc64e12b9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:08.387099 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434f332-3e28-4976-9973-971b09276c38 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:08.423247 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18212c37-4648-4d58-a8b4-bcef98da9ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:08.424583 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18212c37-4648-4d58-a8b4-bcef98da9ef3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.425192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18212c37-4648-4d58-a8b4-bcef98da9ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.425688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18212c37-4648-4d58-a8b4-bcef98da9ef3 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:08.427186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18212c37-4648-4d58-a8b4-bcef98da9ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:08.429503 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18212c37-4648-4d58-a8b4-bcef98da9ef3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:08.431541 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 83/167] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:08 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:08.433127 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee848d-9985-4a1b-9555-697996f79195 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.433657 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee848d-9985-4a1b-9555-697996f79195 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.434162 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee848d-9985-4a1b-9555-697996f79195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:08.457273 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-b3ee848d-9985-4a1b-9555-697996f79195 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:08.459123 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-b3ee848d-9985-4a1b-9555-697996f79195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:08.530329 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 84/168] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:08.531608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee848d-9985-4a1b-9555-697996f79195 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:08.623203 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6291fc-23d1-42ca-84fd-cbcbc64e12b9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:08.634215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee848d-9985-4a1b-9555-697996f79195 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:08.670172 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397e4a6-4161-4e59-958d-e89caac46020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:08.671291 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e4a6-4161-4e59-958d-e89caac46020 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.671819 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e4a6-4161-4e59-958d-e89caac46020 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.672313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397e4a6-4161-4e59-958d-e89caac46020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:08.713872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881f818-b2cf-4c48-b183-b0c11fc341b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:08.714878 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881f818-b2cf-4c48-b183-b0c11fc341b4 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.715171 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881f818-b2cf-4c48-b183-b0c11fc341b4 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.715476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881f818-b2cf-4c48-b183-b0c11fc341b4 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:08.719126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881f818-b2cf-4c48-b183-b0c11fc341b4 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:08.720629 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881f818-b2cf-4c48-b183-b0c11fc341b4 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:08.722455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 85/169] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:08 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:08.723518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881f818-b2cf-4c48-b183-b0c11fc341b4 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:08.748446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397e4a6-4161-4e59-958d-e89caac46020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:08.750373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397e4a6-4161-4e59-958d-e89caac46020 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:08.852062 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 85/170] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:08.853368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397e4a6-4161-4e59-958d-e89caac46020 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:08.959256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397e4a6-4161-4e59-958d-e89caac46020 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:08.987230 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881f818-b2cf-4c48-b183-b0c11fc341b4 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:08.996810 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc43bb3-7152-47e1-9b56-48c3de30e0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:08.997950 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc43bb3-7152-47e1-9b56-48c3de30e0a3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:08.998487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc43bb3-7152-47e1-9b56-48c3de30e0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:08.998995 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc43bb3-7152-47e1-9b56-48c3de30e0a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.010110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc43bb3-7152-47e1-9b56-48c3de30e0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-ddc43bb3-7152-47e1-9b56-48c3de30e0a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:09.018690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:09.019768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:09.020737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:09.023957 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 86/171] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:09.024965 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc43bb3-7152-47e1-9b56-48c3de30e0a3 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:09.066859 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:09.067556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.067854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.068181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.069090 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] RBAC: Adding query filter params (role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b, user.id=2a6b5024341c429e891c0f92594e0c9b) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:09.069425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:09.070950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:09.086965 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 86/172] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:08 2022] GET /identity/v3/role_assignments?user.id=2a6b5024341c429e891c0f92594e0c9b&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 608 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:09.088248 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:09.145559 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc43bb3-7152-47e1-9b56-48c3de30e0a3 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:09.232882 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308af79-a160-45d8-9722-3ff028d5b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:09.233905 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308af79-a160-45d8-9722-3ff028d5b6ce None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.234199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308af79-a160-45d8-9722-3ff028d5b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.234486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308af79-a160-45d8-9722-3ff028d5b6ce None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/users/c187e47f86364d0f9b952b57e6548b83/roles/9178c520b46b41e48fc1659fe0129123` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.242554 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308af79-a160-45d8-9722-3ff028d5b6ce None admin] RBAC: Authorizing `identity:create_grant(project_id=38e69abc282f4bf1aa231ecc15fc213b, user_id=c187e47f86364d0f9b952b57e6548b83, role_id=9178c520b46b41e48fc1659fe0129123)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:09.244045 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308af79-a160-45d8-9722-3ff028d5b6ce None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:09.261884 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f5482-9967-406e-8d85-1206d7bcf735 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:09.279354 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 87/173] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:09 2022] PUT /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/users/c187e47f86364d0f9b952b57e6548b83/roles/9178c520b46b41e48fc1659fe0129123 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:09.280381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308af79-a160-45d8-9722-3ff028d5b6ce None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:09.332370 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcff29-d540-421a-8387-3af23ac839f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:09.334292 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcff29-d540-421a-8387-3af23ac839f1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.334834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcff29-d540-421a-8387-3af23ac839f1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.335341 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcff29-d540-421a-8387-3af23ac839f1 None admin] PATH_INFO: `/v3/projects/4f1c568272f648beab76e72830fde086` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.337006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fcff29-d540-421a-8387-3af23ac839f1 None admin] RBAC: Authorizing `identity:get_project(project_id=4f1c568272f648beab76e72830fde086)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:09.339039 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fcff29-d540-421a-8387-3af23ac839f1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:09.341902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 87/174] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:09 2022] GET /identity/v3/projects/4f1c568272f648beab76e72830fde086 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:09.349086 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcff29-d540-421a-8387-3af23ac839f1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:09.455918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308af79-a160-45d8-9722-3ff028d5b6ce None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:09.498448 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcff29-d540-421a-8387-3af23ac839f1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:09.508966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337827e6-80b3-43b3-be88-0f5bac639419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:09.509740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337827e6-80b3-43b3-be88-0f5bac639419 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.510033 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337827e6-80b3-43b3-be88-0f5bac639419 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.510322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337827e6-80b3-43b3-be88-0f5bac639419 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.511144 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337827e6-80b3-43b3-be88-0f5bac639419 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:09.511463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337827e6-80b3-43b3-be88-0f5bac639419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:09.512875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337827e6-80b3-43b3-be88-0f5bac639419 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:09.522750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 88/175] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:09.524146 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720393f-8022-4a93-8e01-e70d3ad89444 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.524446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720393f-8022-4a93-8e01-e70d3ad89444 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.524733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720393f-8022-4a93-8e01-e70d3ad89444 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.530457 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 89/176] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:09.531646 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720393f-8022-4a93-8e01-e70d3ad89444 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:09.547329 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78011c2a-d261-4670-9893-e6e24cae5143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:09.548016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78011c2a-d261-4670-9893-e6e24cae5143 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.548301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78011c2a-d261-4670-9893-e6e24cae5143 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.548580 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78011c2a-d261-4670-9893-e6e24cae5143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.549477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78011c2a-d261-4670-9893-e6e24cae5143 None admin] RBAC: Adding query filter params (role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=4f1c568272f648beab76e72830fde086, user.id=2a6b5024341c429e891c0f92594e0c9b) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:09.549798 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78011c2a-d261-4670-9893-e6e24cae5143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:09.551284 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78011c2a-d261-4670-9893-e6e24cae5143 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:09.566290 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 88/177] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:09 2022] GET /identity/v3/role_assignments?user.id=2a6b5024341c429e891c0f92594e0c9b&scope.project.id=4f1c568272f648beab76e72830fde086&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:09.567659 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19a04-5f00-4d20-ae3d-f701bcab442a None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.567953 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19a04-5f00-4d20-ae3d-f701bcab442a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.568779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19a04-5f00-4d20-ae3d-f701bcab442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.602463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-3cb19a04-5f00-4d20-ae3d-f701bcab442a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:09.603699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-3cb19a04-5f00-4d20-ae3d-f701bcab442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:09.672705 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720393f-8022-4a93-8e01-e70d3ad89444 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:09.718016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 89/178] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:09.719838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbadb2-be20-4146-a3c8-71810d4f2c3c None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.720134 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbadb2-be20-4146-a3c8-71810d4f2c3c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.720477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbadb2-be20-4146-a3c8-71810d4f2c3c None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.721480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa10f06-f15a-484a-b792-e32e6be516cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:09.721845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 90/179] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:09.722381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa10f06-f15a-484a-b792-e32e6be516cc None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.722675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa10f06-f15a-484a-b792-e32e6be516cc None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.722960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa10f06-f15a-484a-b792-e32e6be516cc None admin] PATH_INFO: `/v3/users/2a6b5024341c429e891c0f92594e0c9b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.724505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa10f06-f15a-484a-b792-e32e6be516cc None admin] RBAC: Authorizing `identity:get_user(user_id=2a6b5024341c429e891c0f92594e0c9b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:09.725999 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa10f06-f15a-484a-b792-e32e6be516cc None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:09.727988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 90/180] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:09 2022] GET /identity/v3/users/2a6b5024341c429e891c0f92594e0c9b => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:09.728300 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872865-4100-4f10-b6e3-de872e796fcf None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.728662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872865-4100-4f10-b6e3-de872e796fcf None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.728960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872865-4100-4f10-b6e3-de872e796fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:09.733188 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa10f06-f15a-484a-b792-e32e6be516cc None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:09.749143 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-30872865-4100-4f10-b6e3-de872e796fcf None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:09.750405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-30872865-4100-4f10-b6e3-de872e796fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:09.849138 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 91/181] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:09.851910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30872865-4100-4f10-b6e3-de872e796fcf None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:09.862414 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa10f06-f15a-484a-b792-e32e6be516cc None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:09.924628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624a491-94de-428b-b5d2-51809e58c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:09.925699 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a491-94de-428b-b5d2-51809e58c5e9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:09.926206 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a491-94de-428b-b5d2-51809e58c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:09.926705 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a491-94de-428b-b5d2-51809e58c5e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.004692 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624a491-94de-428b-b5d2-51809e58c5e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.007631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624a491-94de-428b-b5d2-51809e58c5e9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.009319 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30872865-4100-4f10-b6e3-de872e796fcf None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.051907 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86deb83-c0ba-4caf-957a-5563c799c70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.053679 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86deb83-c0ba-4caf-957a-5563c799c70e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.053956 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86deb83-c0ba-4caf-957a-5563c799c70e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.054238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86deb83-c0ba-4caf-957a-5563c799c70e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.066753 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86deb83-c0ba-4caf-957a-5563c799c70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b86deb83-c0ba-4caf-957a-5563c799c70e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:10.077461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:10.078607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:10.079539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 92/182] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:10.080330 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86deb83-c0ba-4caf-957a-5563c799c70e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.127498 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 91/183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:10.128724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624a491-94de-428b-b5d2-51809e58c5e9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.225382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86deb83-c0ba-4caf-957a-5563c799c70e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.269672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624a491-94de-428b-b5d2-51809e58c5e9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.269861 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a39db-991f-443b-b976-3757778a062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.270523 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a39db-991f-443b-b976-3757778a062b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.270814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a39db-991f-443b-b976-3757778a062b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.271095 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a39db-991f-443b-b976-3757778a062b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.281548 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a39db-991f-443b-b976-3757778a062b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-513a39db-991f-443b-b976-3757778a062b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:10.290460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:10.291501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:10.292566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 93/184] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:10.293381 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a39db-991f-443b-b976-3757778a062b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.316103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.317493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.318178 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.318967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.320251 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:10.320939 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.323231 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.334239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 92/185] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:10.335323 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.404941 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a39db-991f-443b-b976-3757778a062b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.438592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcae3ab-de81-4998-be2c-fe0a1efca139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.439566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcae3ab-de81-4998-be2c-fe0a1efca139 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.439856 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcae3ab-de81-4998-be2c-fe0a1efca139 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.440134 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcae3ab-de81-4998-be2c-fe0a1efca139 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.441370 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcae3ab-de81-4998-be2c-fe0a1efca139 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.442834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcae3ab-de81-4998-be2c-fe0a1efca139 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.444688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 94/186] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:10.445515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865c65b-ef04-4e20-9d55-acf22ae51d1d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.448990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcae3ab-de81-4998-be2c-fe0a1efca139 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.480833 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a938ce-7273-4c17-9e60-a5c5e49c6601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.481558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a938ce-7273-4c17-9e60-a5c5e49c6601 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.481850 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a938ce-7273-4c17-9e60-a5c5e49c6601 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.482138 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a938ce-7273-4c17-9e60-a5c5e49c6601 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.482937 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a938ce-7273-4c17-9e60-a5c5e49c6601 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:10.483247 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a938ce-7273-4c17-9e60-a5c5e49c6601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.484628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a938ce-7273-4c17-9e60-a5c5e49c6601 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.493306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 93/187] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:10.494474 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a78a5-d946-4264-adf1-bf0db52ce426 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.494771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a78a5-d946-4264-adf1-bf0db52ce426 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.495056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a78a5-d946-4264-adf1-bf0db52ce426 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.496618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 94/188] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:10.497004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a78a5-d946-4264-adf1-bf0db52ce426 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.560449 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcae3ab-de81-4998-be2c-fe0a1efca139 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.596563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a14dcd-7c16-4fb8-8553-75df981ebacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.597466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a14dcd-7c16-4fb8-8553-75df981ebacb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.597750 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a14dcd-7c16-4fb8-8553-75df981ebacb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.598278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a14dcd-7c16-4fb8-8553-75df981ebacb None admin] PATH_INFO: `/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.611726 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a14dcd-7c16-4fb8-8553-75df981ebacb None admin] RBAC: Authorizing `identity:get_user(user_id=d6cf6fb0701e4c53bcb332ee77a7ccd1)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.613280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a14dcd-7c16-4fb8-8553-75df981ebacb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.613673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a78a5-d946-4264-adf1-bf0db52ce426 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.615004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 95/189] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:10.615777 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a14dcd-7c16-4fb8-8553-75df981ebacb None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.653953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c713e-9b11-4105-abe7-b50f49d64577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.654723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c713e-9b11-4105-abe7-b50f49d64577 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.655019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c713e-9b11-4105-abe7-b50f49d64577 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.655309 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c713e-9b11-4105-abe7-b50f49d64577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.656150 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c713e-9b11-4105-abe7-b50f49d64577 None admin] RBAC: Adding query filter params (role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b, user.id=2a6b5024341c429e891c0f92594e0c9b) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:10.656473 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c713e-9b11-4105-abe7-b50f49d64577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.657953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c713e-9b11-4105-abe7-b50f49d64577 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.664243 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 95/190] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/role_assignments?user.id=2a6b5024341c429e891c0f92594e0c9b&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=044a2d93445743af91a96573d39f21eb => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:10.665467 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396e89-87d0-4fc1-93ea-ccc05897c633 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.665759 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396e89-87d0-4fc1-93ea-ccc05897c633 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.666046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35396e89-87d0-4fc1-93ea-ccc05897c633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.683072 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-35396e89-87d0-4fc1-93ea-ccc05897c633 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:10.684324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-35396e89-87d0-4fc1-93ea-ccc05897c633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:10.737614 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a14dcd-7c16-4fb8-8553-75df981ebacb None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.756663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 96/191] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:10.757713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35396e89-87d0-4fc1-93ea-ccc05897c633 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.789648 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbb8a4-88eb-4512-b259-71028b58d512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.791089 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbb8a4-88eb-4512-b259-71028b58d512 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.792418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbb8a4-88eb-4512-b259-71028b58d512 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.792959 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbb8a4-88eb-4512-b259-71028b58d512 None admin] PATH_INFO: `/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.794488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbb8a4-88eb-4512-b259-71028b58d512 None admin] RBAC: Authorizing `identity:get_user(user_id=d6cf6fb0701e4c53bcb332ee77a7ccd1)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.796303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbb8a4-88eb-4512-b259-71028b58d512 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.799724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 96/192] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:10.802732 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbb8a4-88eb-4512-b259-71028b58d512 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:10.861951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35396e89-87d0-4fc1-93ea-ccc05897c633 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.898750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff37b2e-29f5-4ee5-8240-a1ea75908bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.899426 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff37b2e-29f5-4ee5-8240-a1ea75908bf5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.899712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff37b2e-29f5-4ee5-8240-a1ea75908bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.899989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff37b2e-29f5-4ee5-8240-a1ea75908bf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.944702 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbb8a4-88eb-4512-b259-71028b58d512 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:10.987606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb059072-3634-4390-ab86-b6e314e1e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:10.988420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb059072-3634-4390-ab86-b6e314e1e6eb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:10.988710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb059072-3634-4390-ab86-b6e314e1e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:10.988991 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb059072-3634-4390-ab86-b6e314e1e6eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:10.990250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb059072-3634-4390-ab86-b6e314e1e6eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.991471 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff37b2e-29f5-4ee5-8240-a1ea75908bf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:10.991718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb059072-3634-4390-ab86-b6e314e1e6eb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.992939 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff37b2e-29f5-4ee5-8240-a1ea75908bf5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:10.993568 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 97/193] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:10.998007 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb059072-3634-4390-ab86-b6e314e1e6eb None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:11.104247 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 97/194] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:11.104815 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb059072-3634-4390-ab86-b6e314e1e6eb None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:11.106082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff37b2e-29f5-4ee5-8240-a1ea75908bf5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:11.141916 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d4af4-bb7f-44f3-b398-80ae3e1eba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:11.142570 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4af4-bb7f-44f3-b398-80ae3e1eba52 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:11.142862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4af4-bb7f-44f3-b398-80ae3e1eba52 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:11.143150 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539d4af4-bb7f-44f3-b398-80ae3e1eba52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:11.199784 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff37b2e-29f5-4ee5-8240-a1ea75908bf5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:11.208572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d4af4-bb7f-44f3-b398-80ae3e1eba52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:11.210696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539d4af4-bb7f-44f3-b398-80ae3e1eba52 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:11.233330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5db35-470e-4b87-a19d-64fa9c57fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:11.234007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5db35-470e-4b87-a19d-64fa9c57fe01 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:11.234300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5db35-470e-4b87-a19d-64fa9c57fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:11.234584 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5db35-470e-4b87-a19d-64fa9c57fe01 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:11.235303 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5db35-470e-4b87-a19d-64fa9c57fe01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:11.236799 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5db35-470e-4b87-a19d-64fa9c57fe01 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:11.265398 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 98/195] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:11 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:11.265398 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5db35-470e-4b87-a19d-64fa9c57fe01 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:11.336856 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 98/196] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:11.343508 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d4af4-bb7f-44f3-b398-80ae3e1eba52 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:11.385435 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5db35-470e-4b87-a19d-64fa9c57fe01 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:11.431307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808bdc0-ce32-4b71-aebb-cfadc5968cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:11.433217 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808bdc0-ce32-4b71-aebb-cfadc5968cab None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:11.433805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808bdc0-ce32-4b71-aebb-cfadc5968cab None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:11.434241 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808bdc0-ce32-4b71-aebb-cfadc5968cab None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:11.436764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808bdc0-ce32-4b71-aebb-cfadc5968cab None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:11.438480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808bdc0-ce32-4b71-aebb-cfadc5968cab None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:11.440925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 99/197] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:11 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:11.449091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808bdc0-ce32-4b71-aebb-cfadc5968cab None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:11.476486 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539d4af4-bb7f-44f3-b398-80ae3e1eba52 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:11.524515 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d291cbb-4559-4413-80bd-d8a439ecab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:11.525431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291cbb-4559-4413-80bd-d8a439ecab8d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:11.525728 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291cbb-4559-4413-80bd-d8a439ecab8d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:11.526010 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291cbb-4559-4413-80bd-d8a439ecab8d None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:11.527533 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291cbb-4559-4413-80bd-d8a439ecab8d None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:11.532269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291cbb-4559-4413-80bd-d8a439ecab8d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:11.532269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 99/198] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:11 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:11.538298 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d291cbb-4559-4413-80bd-d8a439ecab8d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:11.582395 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808bdc0-ce32-4b71-aebb-cfadc5968cab None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:11.629226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6986819-92cc-419f-8c31-7a19801d2d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:11.629446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6986819-92cc-419f-8c31-7a19801d2d81 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:11.629687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6986819-92cc-419f-8c31-7a19801d2d81 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:11.630002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6986819-92cc-419f-8c31-7a19801d2d81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:11.631484 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6986819-92cc-419f-8c31-7a19801d2d81 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=1322e64cad064e79a31ef6484516a1de, user.id=d6cf6fb0701e4c53bcb332ee77a7ccd1) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:11.631804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6986819-92cc-419f-8c31-7a19801d2d81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:11.633926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6986819-92cc-419f-8c31-7a19801d2d81 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:11.640849 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 100/199] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:11 2022] GET /identity/v3/role_assignments?user.id=d6cf6fb0701e4c53bcb332ee77a7ccd1&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=ebff922955b74c919d58613875326151 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:11.662737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d291cbb-4559-4413-80bd-d8a439ecab8d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:11.702287 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4421f7-0038-417b-be08-5661a8456e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:11.702901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4421f7-0038-417b-be08-5661a8456e1e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:11.703213 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4421f7-0038-417b-be08-5661a8456e1e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:11.703505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4421f7-0038-417b-be08-5661a8456e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:11.704439 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4421f7-0038-417b-be08-5661a8456e1e None admin] RBAC: Adding query filter params (role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=1322e64cad064e79a31ef6484516a1de, user.id=d6cf6fb0701e4c53bcb332ee77a7ccd1) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:11.704770 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4421f7-0038-417b-be08-5661a8456e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:11.706380 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4421f7-0038-417b-be08-5661a8456e1e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:11.713608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 100/200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:11 2022] GET /identity/v3/role_assignments?user.id=d6cf6fb0701e4c53bcb332ee77a7ccd1&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=044a2d93445743af91a96573d39f21eb => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:12.017780 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72603dd-6823-4a24-829e-e8d77284e4a2 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:12.017983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72603dd-6823-4a24-829e-e8d77284e4a2 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:12.018270 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72603dd-6823-4a24-829e-e8d77284e4a2 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:12.019933 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 101/201] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:12.026165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde757b-2669-4798-b293-81cc20228beb None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:12.026461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde757b-2669-4798-b293-81cc20228beb None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:12.026749 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde757b-2669-4798-b293-81cc20228beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:12.049449 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-ebde757b-2669-4798-b293-81cc20228beb None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:12.050651 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-ebde757b-2669-4798-b293-81cc20228beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:12.137391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 101/202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:12.173873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72603dd-6823-4a24-829e-e8d77284e4a2 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:12.315577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72603dd-6823-4a24-829e-e8d77284e4a2 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:12.364871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af385794-2a59-4c2d-9144-2b1d6bcf8427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:12.365727 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af385794-2a59-4c2d-9144-2b1d6bcf8427 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:12.366025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af385794-2a59-4c2d-9144-2b1d6bcf8427 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:12.366316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af385794-2a59-4c2d-9144-2b1d6bcf8427 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:12.381881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af385794-2a59-4c2d-9144-2b1d6bcf8427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-af385794-2a59-4c2d-9144-2b1d6bcf8427 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:12.389741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:12.390776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:12.391711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 102/203] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:12.399065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde757b-2669-4798-b293-81cc20228beb None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:12.554450 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde757b-2669-4798-b293-81cc20228beb None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:12.596202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:12.597279 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:12.597782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:12.598564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:12.599593 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] RBAC: Adding query filter params (name=reader) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:12.600278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:12.602091 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:12.618733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 102/204] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:12.623973 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af385794-2a59-4c2d-9144-2b1d6bcf8427 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:12.796428 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af385794-2a59-4c2d-9144-2b1d6bcf8427 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:12.845109 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5431faf-8273-4a00-bff1-65ef53e525b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:12.846011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431faf-8273-4a00-bff1-65ef53e525b8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:12.846357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431faf-8273-4a00-bff1-65ef53e525b8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:12.846590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5431faf-8273-4a00-bff1-65ef53e525b8 None admin] PATH_INFO: `/v3/users/c187e47f86364d0f9b952b57e6548b83` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:12.853771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5431faf-8273-4a00-bff1-65ef53e525b8 None admin] RBAC: Authorizing `identity:get_user(user_id=c187e47f86364d0f9b952b57e6548b83)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:12.855264 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5431faf-8273-4a00-bff1-65ef53e525b8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:12.857174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 103/205] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:12 2022] GET /identity/v3/users/c187e47f86364d0f9b952b57e6548b83 => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:12.862490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:13.014678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce848e4e-7865-4933-9775-0774d1cff043 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:13.060112 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601777a-764e-4a9a-a6b0-ce03d992ab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:13.060776 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601777a-764e-4a9a-a6b0-ce03d992ab5d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:13.061081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601777a-764e-4a9a-a6b0-ce03d992ab5d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:13.061370 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601777a-764e-4a9a-a6b0-ce03d992ab5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:13.154511 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601777a-764e-4a9a-a6b0-ce03d992ab5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:13.155995 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601777a-764e-4a9a-a6b0-ce03d992ab5d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:13.258727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 103/206] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:13.270162 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5431faf-8273-4a00-bff1-65ef53e525b8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:13.384565 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5431faf-8273-4a00-bff1-65ef53e525b8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:13.420750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d317421-5d23-407a-a1f2-5bf28132a1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:13.421593 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d317421-5d23-407a-a1f2-5bf28132a1ae None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:13.421880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d317421-5d23-407a-a1f2-5bf28132a1ae None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:13.422338 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d317421-5d23-407a-a1f2-5bf28132a1ae None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:13.423575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d317421-5d23-407a-a1f2-5bf28132a1ae None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:13.425025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d317421-5d23-407a-a1f2-5bf28132a1ae None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:13.426947 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 104/207] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:13 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:13.433226 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601777a-764e-4a9a-a6b0-ce03d992ab5d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:13.562474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601777a-764e-4a9a-a6b0-ce03d992ab5d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:13.604505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18fb0b-98bf-40bd-885d-2a44625d1f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:13.604505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fb0b-98bf-40bd-885d-2a44625d1f45 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:13.604505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fb0b-98bf-40bd-885d-2a44625d1f45 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:13.604505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fb0b-98bf-40bd-885d-2a44625d1f45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:13.605403 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18fb0b-98bf-40bd-885d-2a44625d1f45 None admin] RBAC: Adding query filter params (role.id=9178c520b46b41e48fc1659fe0129123, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b, user.id=c187e47f86364d0f9b952b57e6548b83) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:13.605564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18fb0b-98bf-40bd-885d-2a44625d1f45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:13.608555 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18fb0b-98bf-40bd-885d-2a44625d1f45 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:13.615206 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 104/208] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:13 2022] GET /identity/v3/role_assignments?user.id=c187e47f86364d0f9b952b57e6548b83&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=9178c520b46b41e48fc1659fe0129123 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:14.100912 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b23bb9-b707-49ac-86d7-ac4524e51741 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.101350 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b23bb9-b707-49ac-86d7-ac4524e51741 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.101542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b23bb9-b707-49ac-86d7-ac4524e51741 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.103212 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 105/209] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:14.109513 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2b5e5-6b6d-4e56-9971-caadac438c42 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.109807 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2b5e5-6b6d-4e56-9971-caadac438c42 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.110087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2b5e5-6b6d-4e56-9971-caadac438c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.128488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-20b2b5e5-6b6d-4e56-9971-caadac438c42 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:14.129707 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-20b2b5e5-6b6d-4e56-9971-caadac438c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:14.218644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 105/210] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:14.246080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b23bb9-b707-49ac-86d7-ac4524e51741 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:14.376182 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b23bb9-b707-49ac-86d7-ac4524e51741 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:14.410023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49463f92-7f6e-4cc6-b644-ffc55f9a6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:14.410832 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49463f92-7f6e-4cc6-b644-ffc55f9a6b52 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.411120 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49463f92-7f6e-4cc6-b644-ffc55f9a6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.411402 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49463f92-7f6e-4cc6-b644-ffc55f9a6b52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.422509 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49463f92-7f6e-4cc6-b644-ffc55f9a6b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-49463f92-7f6e-4cc6-b644-ffc55f9a6b52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:14.430664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:14.432411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:14.433587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:14.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ab431-6611-4f30-a57a-d899038740a8 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ab431-6611-4f30-a57a-d899038740a8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.434466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ab431-6611-4f30-a57a-d899038740a8 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.436516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 106/211] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:41:14.436659 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 106/212] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:14.441385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49463f92-7f6e-4cc6-b644-ffc55f9a6b52 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:14.441743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863b423-f73f-4431-bd05-ef9d77a4abca None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.442041 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863b423-f73f-4431-bd05-ef9d77a4abca None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.442328 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863b423-f73f-4431-bd05-ef9d77a4abca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.459551 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-3863b423-f73f-4431-bd05-ef9d77a4abca None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:14.460748 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-3863b423-f73f-4431-bd05-ef9d77a4abca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:14.527951 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 107/213] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:14.529387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d829-a76e-4edd-beb2-422525280b01 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.529689 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d829-a76e-4edd-beb2-422525280b01 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.529974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d829-a76e-4edd-beb2-422525280b01 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.531396 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 108/214] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:14.532409 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144de7c6-b50e-46fc-b671-85eaffeabd30 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.532845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144de7c6-b50e-46fc-b671-85eaffeabd30 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.533157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144de7c6-b50e-46fc-b671-85eaffeabd30 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.537323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 109/215] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:14.540254 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500818-34c9-4d1d-a4d9-1123d11ef373 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.540546 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500818-34c9-4d1d-a4d9-1123d11ef373 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.540827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500818-34c9-4d1d-a4d9-1123d11ef373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.548894 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49463f92-7f6e-4cc6-b644-ffc55f9a6b52 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:14.557690 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-78500818-34c9-4d1d-a4d9-1123d11ef373 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:14.558915 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-78500818-34c9-4d1d-a4d9-1123d11ef373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:14.581058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be801436-6734-4731-bdfb-6aab18dc8e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:14.581743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be801436-6734-4731-bdfb-6aab18dc8e2b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.582032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be801436-6734-4731-bdfb-6aab18dc8e2b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.582318 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be801436-6734-4731-bdfb-6aab18dc8e2b None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.583074 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be801436-6734-4731-bdfb-6aab18dc8e2b None admin] RBAC: Adding query filter params (name=member) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:14.583392 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be801436-6734-4731-bdfb-6aab18dc8e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:14.585335 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be801436-6734-4731-bdfb-6aab18dc8e2b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:14.593183 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 107/216] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:14.594357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35e617-1ea9-40de-af69-9ed86e47ed68 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.594652 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35e617-1ea9-40de-af69-9ed86e47ed68 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.594937 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35e617-1ea9-40de-af69-9ed86e47ed68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.611186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-de35e617-1ea9-40de-af69-9ed86e47ed68 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:14.612399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-de35e617-1ea9-40de-af69-9ed86e47ed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:14.623152 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 110/217] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:14.623995 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78500818-34c9-4d1d-a4d9-1123d11ef373 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:14.688958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 108/218] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:14.689834 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35e617-1ea9-40de-af69-9ed86e47ed68 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:14.721938 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78500818-34c9-4d1d-a4d9-1123d11ef373 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:14.754245 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13e295-6c7e-4726-b7da-6aecb20a9a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:14.755037 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13e295-6c7e-4726-b7da-6aecb20a9a7a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.755327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13e295-6c7e-4726-b7da-6aecb20a9a7a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.756136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13e295-6c7e-4726-b7da-6aecb20a9a7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.766146 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13e295-6c7e-4726-b7da-6aecb20a9a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-6f13e295-6c7e-4726-b7da-6aecb20a9a7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:14.774425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:14.775574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 111/219] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:14.781474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13e295-6c7e-4726-b7da-6aecb20a9a7a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:14.786859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35e617-1ea9-40de-af69-9ed86e47ed68 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:14.818918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc4ab9-fd25-437a-a591-de26c5b1f3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:14.819788 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc4ab9-fd25-437a-a591-de26c5b1f3f6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.820075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc4ab9-fd25-437a-a591-de26c5b1f3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.820350 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc4ab9-fd25-437a-a591-de26c5b1f3f6 None admin] PATH_INFO: `/v3/users/8fd3998a8fca46c3b89a295d586dc35d` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.833070 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc4ab9-fd25-437a-a591-de26c5b1f3f6 None admin] RBAC: Authorizing `identity:get_user(user_id=8fd3998a8fca46c3b89a295d586dc35d)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:14.835147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc4ab9-fd25-437a-a591-de26c5b1f3f6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:14.836837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 109/220] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:14 2022] GET /identity/v3/users/8fd3998a8fca46c3b89a295d586dc35d => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:14.837637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc4ab9-fd25-437a-a591-de26c5b1f3f6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:14.870750 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13e295-6c7e-4726-b7da-6aecb20a9a7a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:14.905181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0ac81-1a91-44a7-9475-a4f35238b9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:14.905181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0ac81-1a91-44a7-9475-a4f35238b9c4 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.905181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0ac81-1a91-44a7-9475-a4f35238b9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.905181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0ac81-1a91-44a7-9475-a4f35238b9c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.905574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0ac81-1a91-44a7-9475-a4f35238b9c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:14.906691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0ac81-1a91-44a7-9475-a4f35238b9c4 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:14.908311 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 112/221] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:14.909100 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0ac81-1a91-44a7-9475-a4f35238b9c4 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:14.935086 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc4ab9-fd25-437a-a591-de26c5b1f3f6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:14.967919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cead7-d15c-410f-bbe3-6f16995aee87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:14.968734 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cead7-d15c-410f-bbe3-6f16995aee87 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:14.969025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cead7-d15c-410f-bbe3-6f16995aee87 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:14.969334 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90cead7-d15c-410f-bbe3-6f16995aee87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:14.981349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90cead7-d15c-410f-bbe3-6f16995aee87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-b90cead7-d15c-410f-bbe3-6f16995aee87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:14.993709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:14.994768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 110/222] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:14.995745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cead7-d15c-410f-bbe3-6f16995aee87 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.015445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0ac81-1a91-44a7-9475-a4f35238b9c4 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.046811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.047411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.047697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.047978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.048730 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:15.049120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.050441 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.058692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 113/223] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:15.058692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.094561 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90cead7-d15c-410f-bbe3-6f16995aee87 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.126657 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de9b3-2d62-4137-aad4-bc0167a69ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.127311 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de9b3-2d62-4137-aad4-bc0167a69ff7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.127605 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de9b3-2d62-4137-aad4-bc0167a69ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.127888 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de9b3-2d62-4137-aad4-bc0167a69ff7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.156456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e1a25-8e7b-400b-b9ac-6cd565f1cd29 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.188076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85484670-dfdf-4ffc-8034-5ae7a48b403a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.188734 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85484670-dfdf-4ffc-8034-5ae7a48b403a None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.189021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85484670-dfdf-4ffc-8034-5ae7a48b403a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.189332 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85484670-dfdf-4ffc-8034-5ae7a48b403a None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.190037 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85484670-dfdf-4ffc-8034-5ae7a48b403a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.191422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13de9b3-2d62-4137-aad4-bc0167a69ff7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.191531 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85484670-dfdf-4ffc-8034-5ae7a48b403a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.192897 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13de9b3-2d62-4137-aad4-bc0167a69ff7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.210954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 114/224] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:15 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:15.211793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85484670-dfdf-4ffc-8034-5ae7a48b403a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.303886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 111/225] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:15.303886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de9b3-2d62-4137-aad4-bc0167a69ff7 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.319005 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85484670-dfdf-4ffc-8034-5ae7a48b403a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.359850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.360588 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.360882 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.361199 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.361976 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:15.362293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.363717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.377194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 115/226] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:15.377194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.415684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de9b3-2d62-4137-aad4-bc0167a69ff7 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.451329 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da669-6097-4307-b712-a21c2a65ebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.452222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da669-6097-4307-b712-a21c2a65ebc6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.452506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da669-6097-4307-b712-a21c2a65ebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.452784 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5da669-6097-4307-b712-a21c2a65ebc6 None admin] PATH_INFO: `/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.456688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5da669-6097-4307-b712-a21c2a65ebc6 None admin] RBAC: Authorizing `identity:get_user(user_id=d6cf6fb0701e4c53bcb332ee77a7ccd1)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.456688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5da669-6097-4307-b712-a21c2a65ebc6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.457556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 112/227] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:15 2022] GET /identity/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:15.458284 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da669-6097-4307-b712-a21c2a65ebc6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.482280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03699d1-f7ec-4fc8-8986-64e324d7ec35 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.517809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f83c4-212a-4011-aea0-c2825b52c9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.518694 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f83c4-212a-4011-aea0-c2825b52c9b6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.518990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f83c4-212a-4011-aea0-c2825b52c9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.519274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f83c4-212a-4011-aea0-c2825b52c9b6 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.520620 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f83c4-212a-4011-aea0-c2825b52c9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.522796 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f83c4-212a-4011-aea0-c2825b52c9b6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.524793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 116/228] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:15 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:15.525532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f83c4-212a-4011-aea0-c2825b52c9b6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.564133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5da669-6097-4307-b712-a21c2a65ebc6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.599268 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69430df4-7945-4082-9966-74fe12da2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.600161 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69430df4-7945-4082-9966-74fe12da2ffb None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.600452 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69430df4-7945-4082-9966-74fe12da2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.600748 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69430df4-7945-4082-9966-74fe12da2ffb None admin] PATH_INFO: `/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.602333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69430df4-7945-4082-9966-74fe12da2ffb None admin] RBAC: Authorizing `identity:get_user(user_id=d6cf6fb0701e4c53bcb332ee77a7ccd1)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.603475 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69430df4-7945-4082-9966-74fe12da2ffb None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.605623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 113/229] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:15 2022] GET /identity/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:15.606390 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69430df4-7945-4082-9966-74fe12da2ffb None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.639297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f83c4-212a-4011-aea0-c2825b52c9b6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.680435 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe10132-4b19-443b-83a6-560777ad4c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.681445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe10132-4b19-443b-83a6-560777ad4c25 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.681774 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe10132-4b19-443b-83a6-560777ad4c25 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.682060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe10132-4b19-443b-83a6-560777ad4c25 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.683291 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe10132-4b19-443b-83a6-560777ad4c25 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.684745 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe10132-4b19-443b-83a6-560777ad4c25 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.686504 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 117/230] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:15 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:15.687289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe10132-4b19-443b-83a6-560777ad4c25 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.712528 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69430df4-7945-4082-9966-74fe12da2ffb None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.748579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.749323 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.749634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.750229 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.751440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] RBAC: Adding query filter params (role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=1322e64cad064e79a31ef6484516a1de, user.id=8fd3998a8fca46c3b89a295d586dc35d) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:15.751771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.753730 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.762361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 114/231] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:15 2022] GET /identity/v3/role_assignments?user.id=8fd3998a8fca46c3b89a295d586dc35d&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:15.763706 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.803961 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe10132-4b19-443b-83a6-560777ad4c25 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.840862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe017f2-2e2f-49c4-8589-1f5cd7a23f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.842191 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe017f2-2e2f-49c4-8589-1f5cd7a23f9f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.842719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe017f2-2e2f-49c4-8589-1f5cd7a23f9f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.843222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe017f2-2e2f-49c4-8589-1f5cd7a23f9f None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.844660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe017f2-2e2f-49c4-8589-1f5cd7a23f9f None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.846367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe017f2-2e2f-49c4-8589-1f5cd7a23f9f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:15.848606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 118/232] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:15 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:15.858078 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe017f2-2e2f-49c4-8589-1f5cd7a23f9f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:15.890997 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6991678-ade0-41bb-8c4c-072216bd7137 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:15.961843 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b24226-45e1-48c4-9724-0510fdd38d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:15.962782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b24226-45e1-48c4-9724-0510fdd38d71 None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:15.963076 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b24226-45e1-48c4-9724-0510fdd38d71 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:15.963363 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b24226-45e1-48c4-9724-0510fdd38d71 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de/users/d6cf6fb0701e4c53bcb332ee77a7ccd1/roles/ebff922955b74c919d58613875326151` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:15.969904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b24226-45e1-48c4-9724-0510fdd38d71 None admin] RBAC: Authorizing `identity:create_grant(project_id=1322e64cad064e79a31ef6484516a1de, user_id=d6cf6fb0701e4c53bcb332ee77a7ccd1, role_id=ebff922955b74c919d58613875326151)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:15.971991 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b24226-45e1-48c4-9724-0510fdd38d71 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:16.002102 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 115/233] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:15 2022] PUT /identity/v3/projects/1322e64cad064e79a31ef6484516a1de/users/d6cf6fb0701e4c53bcb332ee77a7ccd1/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:16.048767 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe017f2-2e2f-49c4-8589-1f5cd7a23f9f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:16.092837 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89655c-3ef3-47c3-875b-f0da04ca0dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:16.093704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89655c-3ef3-47c3-875b-f0da04ca0dc8 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:16.094000 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89655c-3ef3-47c3-875b-f0da04ca0dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:16.094290 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89655c-3ef3-47c3-875b-f0da04ca0dc8 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de/users/d6cf6fb0701e4c53bcb332ee77a7ccd1/roles/044a2d93445743af91a96573d39f21eb` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:16.096585 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89655c-3ef3-47c3-875b-f0da04ca0dc8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1322e64cad064e79a31ef6484516a1de, user_id=d6cf6fb0701e4c53bcb332ee77a7ccd1, role_id=044a2d93445743af91a96573d39f21eb)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:16.098406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89655c-3ef3-47c3-875b-f0da04ca0dc8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:16.123926 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 119/234] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:15 2022] PUT /identity/v3/projects/1322e64cad064e79a31ef6484516a1de/users/d6cf6fb0701e4c53bcb332ee77a7ccd1/roles/044a2d93445743af91a96573d39f21eb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:17.956930 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bbc94-d24f-485a-95a4-9a7596188db9 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:17.957968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bbc94-d24f-485a-95a4-9a7596188db9 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:17.960374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911bbc94-d24f-485a-95a4-9a7596188db9 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:17.962261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 116/235] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:17.968032 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c305ca-15c5-4655-9a3d-56ba48932486 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:17.968880 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c305ca-15c5-4655-9a3d-56ba48932486 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:17.969398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c305ca-15c5-4655-9a3d-56ba48932486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:17.971129 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cbce-a83a-474c-828a-406e038bca4e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:17.971645 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cbce-a83a-474c-828a-406e038bca4e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:17.972148 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cbce-a83a-474c-828a-406e038bca4e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:17.974252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 117/236] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:17.979448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93009-3599-42b8-8a39-38e97fb3d22b None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:17.979747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93009-3599-42b8-8a39-38e97fb3d22b None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:17.980037 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93009-3599-42b8-8a39-38e97fb3d22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:18.010908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-66c305ca-15c5-4655-9a3d-56ba48932486 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:18.012137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-66c305ca-15c5-4655-9a3d-56ba48932486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:18.015134 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-bfc93009-3599-42b8-8a39-38e97fb3d22b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:18.016319 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-bfc93009-3599-42b8-8a39-38e97fb3d22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:18.154344 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 118/237] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:18.156548 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 120/238] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:18.175328 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc93009-3599-42b8-8a39-38e97fb3d22b None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:18.212872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c305ca-15c5-4655-9a3d-56ba48932486 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:18.371009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c305ca-15c5-4655-9a3d-56ba48932486 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:18.386584 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc93009-3599-42b8-8a39-38e97fb3d22b None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:18.413070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e88be-6d82-4f08-b650-f8e4e322d84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:18.413920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e88be-6d82-4f08-b650-f8e4e322d84b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:18.414208 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e88be-6d82-4f08-b650-f8e4e322d84b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:18.414488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e88be-6d82-4f08-b650-f8e4e322d84b None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:18.417750 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e88be-6d82-4f08-b650-f8e4e322d84b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:18.419233 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e88be-6d82-4f08-b650-f8e4e322d84b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:18.421109 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 121/239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:18.428100 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e88be-6d82-4f08-b650-f8e4e322d84b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:18.453764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46e822-a009-4986-b5af-f31a4c5c21c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:18.454597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46e822-a009-4986-b5af-f31a4c5c21c5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:18.454896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46e822-a009-4986-b5af-f31a4c5c21c5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:18.455188 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46e822-a009-4986-b5af-f31a4c5c21c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:18.475279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46e822-a009-4986-b5af-f31a4c5c21c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-1b46e822-a009-4986-b5af-f31a4c5c21c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:18.492710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:18.493813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 119/240] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:18.494744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ed2d-3f4b-4852-aa1d-0c9421f3c8e7 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:18.495571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ed2d-3f4b-4852-aa1d-0c9421f3c8e7 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:18.495571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ed2d-3f4b-4852-aa1d-0c9421f3c8e7 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:18.500560 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 120/241] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:18.501424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75ed2d-3f4b-4852-aa1d-0c9421f3c8e7 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:18.538934 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e88be-6d82-4f08-b650-f8e4e322d84b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:18.579901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522b9ea-a5cd-49f8-ba4b-a05daaa43f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:18.580759 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522b9ea-a5cd-49f8-ba4b-a05daaa43f5f None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:18.581065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522b9ea-a5cd-49f8-ba4b-a05daaa43f5f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:18.581630 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522b9ea-a5cd-49f8-ba4b-a05daaa43f5f None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:18.582427 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522b9ea-a5cd-49f8-ba4b-a05daaa43f5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:18.585070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522b9ea-a5cd-49f8-ba4b-a05daaa43f5f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:18.607436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 122/242] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:18 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:18.608953 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa4cc5-0d00-455d-b28a-b2695c52d534 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:18.609272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa4cc5-0d00-455d-b28a-b2695c52d534 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:18.609556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa4cc5-0d00-455d-b28a-b2695c52d534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:18.616189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75ed2d-3f4b-4852-aa1d-0c9421f3c8e7 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:18.628970 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-1dfa4cc5-0d00-455d-b28a-b2695c52d534 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:18.630786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-1dfa4cc5-0d00-455d-b28a-b2695c52d534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:18.653779 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:18.654534 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:18.654826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:18.655114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:18.655896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] RBAC: Adding query filter params (name=reader) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:18.656215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:18.657730 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:18.668529 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 121/243] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:18.674011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:18.706160 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 123/244] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:18.805280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa4cc5-0d00-455d-b28a-b2695c52d534 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:18.820990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66a530-f82f-43a6-87fb-fd757d48fdde None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:18.861737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadf92c-8eb7-485e-bc40-72e3db8f40a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:18.862648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadf92c-8eb7-485e-bc40-72e3db8f40a3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:18.862936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadf92c-8eb7-485e-bc40-72e3db8f40a3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:18.863223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadf92c-8eb7-485e-bc40-72e3db8f40a3 None admin] PATH_INFO: `/v3/users/d7a99989198e407089cff609e8765e92` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:18.887572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadf92c-8eb7-485e-bc40-72e3db8f40a3 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a99989198e407089cff609e8765e92)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:18.889147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadf92c-8eb7-485e-bc40-72e3db8f40a3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:18.891496 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 122/245] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:18 2022] GET /identity/v3/users/d7a99989198e407089cff609e8765e92 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:18.898405 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadf92c-8eb7-485e-bc40-72e3db8f40a3 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:18.971147 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa4cc5-0d00-455d-b28a-b2695c52d534 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.007090 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadf92c-8eb7-485e-bc40-72e3db8f40a3 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.009698 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98774f6-7926-4886-8dfa-84a3a284a61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.010520 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98774f6-7926-4886-8dfa-84a3a284a61f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.010809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98774f6-7926-4886-8dfa-84a3a284a61f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.011093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98774f6-7926-4886-8dfa-84a3a284a61f None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.025495 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98774f6-7926-4886-8dfa-84a3a284a61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-d98774f6-7926-4886-8dfa-84a3a284a61f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:19.031369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:19.032417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:19.033432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 124/246] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:19.038486 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973ee1f-e0ee-4029-81a5-7b4734cd0f8b None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.038848 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973ee1f-e0ee-4029-81a5-7b4734cd0f8b None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.039295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973ee1f-e0ee-4029-81a5-7b4734cd0f8b None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.040817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 125/247] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:19.041461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973ee1f-e0ee-4029-81a5-7b4734cd0f8b None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.047334 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c0ee2-893a-4b60-93f7-e076af59db08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.048014 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c0ee2-893a-4b60-93f7-e076af59db08 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.048308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c0ee2-893a-4b60-93f7-e076af59db08 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.048594 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c0ee2-893a-4b60-93f7-e076af59db08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.120955 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c0ee2-893a-4b60-93f7-e076af59db08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:19.122492 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c0ee2-893a-4b60-93f7-e076af59db08 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:19.152228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973ee1f-e0ee-4029-81a5-7b4734cd0f8b None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.188732 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cce261-0def-425e-8fd0-faf8eaa1d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.190298 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce261-0def-425e-8fd0-faf8eaa1d0ff None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.191020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce261-0def-425e-8fd0-faf8eaa1d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.191744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cce261-0def-425e-8fd0-faf8eaa1d0ff None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.192811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cce261-0def-425e-8fd0-faf8eaa1d0ff None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:19.193154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cce261-0def-425e-8fd0-faf8eaa1d0ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:19.194539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cce261-0def-425e-8fd0-faf8eaa1d0ff None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:19.204279 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 126/248] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:19.205607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16649d12-e8c1-4b5a-bf2b-1416d397a79a None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.205896 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16649d12-e8c1-4b5a-bf2b-1416d397a79a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.206180 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16649d12-e8c1-4b5a-bf2b-1416d397a79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.222428 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-16649d12-e8c1-4b5a-bf2b-1416d397a79a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:19.223683 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 123/249] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:19.223833 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-16649d12-e8c1-4b5a-bf2b-1416d397a79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:19.225687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c96d3-057a-4414-a363-436d69dcb85e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.225984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c96d3-057a-4414-a363-436d69dcb85e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.226266 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c96d3-057a-4414-a363-436d69dcb85e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.227824 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 124/250] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:19.228597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c96d3-057a-4414-a363-436d69dcb85e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.292844 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 127/251] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:19.293791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16649d12-e8c1-4b5a-bf2b-1416d397a79a None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.334384 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c96d3-057a-4414-a363-436d69dcb85e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.370467 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d00fa-5c8f-4c8e-bd15-2bb7d1895a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.371354 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d00fa-5c8f-4c8e-bd15-2bb7d1895a00 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.371639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d00fa-5c8f-4c8e-bd15-2bb7d1895a00 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.371923 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d00fa-5c8f-4c8e-bd15-2bb7d1895a00 None admin] PATH_INFO: `/v3/users/8fd3998a8fca46c3b89a295d586dc35d` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.373162 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d00fa-5c8f-4c8e-bd15-2bb7d1895a00 None admin] RBAC: Authorizing `identity:get_user(user_id=8fd3998a8fca46c3b89a295d586dc35d)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:19.374623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d00fa-5c8f-4c8e-bd15-2bb7d1895a00 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:19.376576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 125/252] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/users/8fd3998a8fca46c3b89a295d586dc35d => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:19.377793 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f46829-a40c-48ee-b1c2-ac2e033aaf45 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.378080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f46829-a40c-48ee-b1c2-ac2e033aaf45 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.378361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f46829-a40c-48ee-b1c2-ac2e033aaf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.395145 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-e8f46829-a40c-48ee-b1c2-ac2e033aaf45 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:19.396384 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-e8f46829-a40c-48ee-b1c2-ac2e033aaf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:19.397741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16649d12-e8c1-4b5a-bf2b-1416d397a79a None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.434183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f1377-ba39-46d1-a327-1272a47e0982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.435107 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f1377-ba39-46d1-a327-1272a47e0982 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.435390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f1377-ba39-46d1-a327-1272a47e0982 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.435667 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f1377-ba39-46d1-a327-1272a47e0982 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.436954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f1377-ba39-46d1-a327-1272a47e0982 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:19.438506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f1377-ba39-46d1-a327-1272a47e0982 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:19.440717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 128/253] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:19.441599 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f1377-ba39-46d1-a327-1272a47e0982 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.473280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 126/254] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:19.474791 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f46829-a40c-48ee-b1c2-ac2e033aaf45 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.560725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f1377-ba39-46d1-a327-1272a47e0982 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.598464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e639d-091d-4759-9b09-58f97fd80015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.599998 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e639d-091d-4759-9b09-58f97fd80015 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.600284 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e639d-091d-4759-9b09-58f97fd80015 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.600558 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e639d-091d-4759-9b09-58f97fd80015 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.602439 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f46829-a40c-48ee-b1c2-ac2e033aaf45 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.611141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e639d-091d-4759-9b09-58f97fd80015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-d82e639d-091d-4759-9b09-58f97fd80015 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:19.619027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:19.620075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 129/255] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:19.621743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e639d-091d-4759-9b09-58f97fd80015 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.641846 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1edd92-faff-4368-a197-a6fa3baccaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.643071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1edd92-faff-4368-a197-a6fa3baccaa5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.643592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1edd92-faff-4368-a197-a6fa3baccaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.644108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1edd92-faff-4368-a197-a6fa3baccaa5 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.645570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1edd92-faff-4368-a197-a6fa3baccaa5 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:19.647795 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1edd92-faff-4368-a197-a6fa3baccaa5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:19.649909 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 127/256] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:19.651075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1edd92-faff-4368-a197-a6fa3baccaa5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.733017 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e639d-091d-4759-9b09-58f97fd80015 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.753609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1edd92-faff-4368-a197-a6fa3baccaa5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.767543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.768216 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.768506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.768790 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.769678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] RBAC: Adding query filter params (role.id=9178c520b46b41e48fc1659fe0129123, scope.project.id=1322e64cad064e79a31ef6484516a1de, user.id=d7a99989198e407089cff609e8765e92) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:19.770061 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:19.772131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:19.778289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 130/257] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/role_assignments?user.id=d7a99989198e407089cff609e8765e92&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=9178c520b46b41e48fc1659fe0129123 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:19.779200 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.793967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69a7ba-de7f-4ce3-ba04-489c14ce1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.795168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69a7ba-de7f-4ce3-ba04-489c14ce1acb None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.795675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69a7ba-de7f-4ce3-ba04-489c14ce1acb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.796171 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69a7ba-de7f-4ce3-ba04-489c14ce1acb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.808296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69a7ba-de7f-4ce3-ba04-489c14ce1acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-3b69a7ba-de7f-4ce3-ba04-489c14ce1acb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:19.817075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:19.818192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:19.819236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:19.822165 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 128/258] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:19.823296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69a7ba-de7f-4ce3-ba04-489c14ce1acb None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.894116 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1965c6-3252-4571-9ae5-a2d10d13e0cc None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.931622 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.932310 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.932601 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.932886 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.933693 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:19.934009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:19.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69a7ba-de7f-4ce3-ba04-489c14ce1acb None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:19.935409 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:19.945884 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 131/259] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:19.946826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:19.974764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921203a-1832-4357-9c80-d37d5f07f59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:19.976060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921203a-1832-4357-9c80-d37d5f07f59d None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:19.976593 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921203a-1832-4357-9c80-d37d5f07f59d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:19.977136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921203a-1832-4357-9c80-d37d5f07f59d None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de/users/8fd3998a8fca46c3b89a295d586dc35d/roles/22b9b0cd5e41452380f6aff61caa9f91` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:19.979415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921203a-1832-4357-9c80-d37d5f07f59d None admin] RBAC: Authorizing `identity:create_grant(project_id=1322e64cad064e79a31ef6484516a1de, user_id=8fd3998a8fca46c3b89a295d586dc35d, role_id=22b9b0cd5e41452380f6aff61caa9f91)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:19.981140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921203a-1832-4357-9c80-d37d5f07f59d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:20.006925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 129/260] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:19 2022] PUT /identity/v3/projects/1322e64cad064e79a31ef6484516a1de/users/8fd3998a8fca46c3b89a295d586dc35d/roles/22b9b0cd5e41452380f6aff61caa9f91 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:20.010306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921203a-1832-4357-9c80-d37d5f07f59d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:20.140187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538ca60-a4a1-4a62-a4ab-4d8f1e5e3f17 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:20.145361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921203a-1832-4357-9c80-d37d5f07f59d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:20.187241 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928dea8-b565-4c5e-9ad1-d44571332006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:20.188160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928dea8-b565-4c5e-9ad1-d44571332006 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:20.188458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928dea8-b565-4c5e-9ad1-d44571332006 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:20.188924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928dea8-b565-4c5e-9ad1-d44571332006 None admin] PATH_INFO: `/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:20.190743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928dea8-b565-4c5e-9ad1-d44571332006 None admin] RBAC: Authorizing `identity:get_user(user_id=d6cf6fb0701e4c53bcb332ee77a7ccd1)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:20.192439 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928dea8-b565-4c5e-9ad1-d44571332006 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:20.195483 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 130/261] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:20 2022] GET /identity/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:20.207167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928dea8-b565-4c5e-9ad1-d44571332006 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:20.224503 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:20.225245 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:20.225538 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:20.225825 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:20.226630 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:20.226950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:20.228376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:20.253365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 132/262] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:20.259804 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:20.380032 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbd043-1d45-4261-85e6-89dcf3495ca0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:20.412766 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928dea8-b565-4c5e-9ad1-d44571332006 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:20.414484 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d36b8e-e335-4e2c-892e-0028492058f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:20.415375 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d36b8e-e335-4e2c-892e-0028492058f0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:20.415663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d36b8e-e335-4e2c-892e-0028492058f0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:20.415944 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d36b8e-e335-4e2c-892e-0028492058f0 None admin] PATH_INFO: `/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:20.417200 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d36b8e-e335-4e2c-892e-0028492058f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d6cf6fb0701e4c53bcb332ee77a7ccd1)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:20.418676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d36b8e-e335-4e2c-892e-0028492058f0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:20.420708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 133/263] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:20 2022] GET /identity/v3/users/d6cf6fb0701e4c53bcb332ee77a7ccd1 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:20.428135 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d36b8e-e335-4e2c-892e-0028492058f0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:20.477342 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ceb75-2c07-4260-aec4-77dfa180847e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:20.478019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ceb75-2c07-4260-aec4-77dfa180847e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:20.478313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ceb75-2c07-4260-aec4-77dfa180847e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:20.478600 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ceb75-2c07-4260-aec4-77dfa180847e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:20.597019 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d36b8e-e335-4e2c-892e-0028492058f0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:20.606227 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ceb75-2c07-4260-aec4-77dfa180847e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:20.607737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ceb75-2c07-4260-aec4-77dfa180847e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:20.662860 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6c4b8-aabf-4d91-b95c-4e04278084a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:20.663526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6c4b8-aabf-4d91-b95c-4e04278084a2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:20.663817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6c4b8-aabf-4d91-b95c-4e04278084a2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:20.664102 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6c4b8-aabf-4d91-b95c-4e04278084a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:20.757103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 131/264] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:20.761119 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ceb75-2c07-4260-aec4-77dfa180847e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:20.789777 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6c4b8-aabf-4d91-b95c-4e04278084a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:20.791282 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6c4b8-aabf-4d91-b95c-4e04278084a2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:20.914024 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ceb75-2c07-4260-aec4-77dfa180847e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:20.918283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 134/265] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:41:20.924730 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6c4b8-aabf-4d91-b95c-4e04278084a2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:20.949611 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91046d9-c1db-461e-93ea-e961bca1b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:20.950860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91046d9-c1db-461e-93ea-e961bca1b6ab None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:20.951345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91046d9-c1db-461e-93ea-e961bca1b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:20.951817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91046d9-c1db-461e-93ea-e961bca1b6ab None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:20.953320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91046d9-c1db-461e-93ea-e961bca1b6ab None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:20.955272 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91046d9-c1db-461e-93ea-e961bca1b6ab None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:20.957955 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 132/266] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:20 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:20.963405 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91046d9-c1db-461e-93ea-e961bca1b6ab None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:21.079722 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91046d9-c1db-461e-93ea-e961bca1b6ab None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:21.119713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:21.120885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:21.121416 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:21.121893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:21.122502 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6c4b8-aabf-4d91-b95c-4e04278084a2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:21.123186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=1322e64cad064e79a31ef6484516a1de, user.id=d6cf6fb0701e4c53bcb332ee77a7ccd1) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:21.123713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:21.125613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:21.133067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 133/267] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:20 2022] GET /identity/v3/role_assignments?user.id=d6cf6fb0701e4c53bcb332ee77a7ccd1&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=ebff922955b74c919d58613875326151 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:21.173133 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d182f-de29-4410-a501-7edd7be2a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:21.173955 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d182f-de29-4410-a501-7edd7be2a2bb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:21.174241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d182f-de29-4410-a501-7edd7be2a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:21.174524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d182f-de29-4410-a501-7edd7be2a2bb None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:21.182072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d182f-de29-4410-a501-7edd7be2a2bb None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:21.183590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d182f-de29-4410-a501-7edd7be2a2bb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:21.185715 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 135/268] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:20 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:21.192126 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:21.337145 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959a6ff-d61c-44c6-a9f7-36c5805c90af None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:21.381161 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262438d6-b16a-458d-aaeb-fe6b47475825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:21.382219 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262438d6-b16a-458d-aaeb-fe6b47475825 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:21.382727 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262438d6-b16a-458d-aaeb-fe6b47475825 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:21.383253 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262438d6-b16a-458d-aaeb-fe6b47475825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:21.384322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262438d6-b16a-458d-aaeb-fe6b47475825 None admin] RBAC: Adding query filter params (role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=1322e64cad064e79a31ef6484516a1de, user.id=d6cf6fb0701e4c53bcb332ee77a7ccd1) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:21.384871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262438d6-b16a-458d-aaeb-fe6b47475825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:21.386576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262438d6-b16a-458d-aaeb-fe6b47475825 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:21.393955 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 134/269] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:21 2022] GET /identity/v3/role_assignments?user.id=d6cf6fb0701e4c53bcb332ee77a7ccd1&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=044a2d93445743af91a96573d39f21eb => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:21.805419 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce390c7-eda6-4096-abc6-d5e7f10df3f2 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:21.805832 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce390c7-eda6-4096-abc6-d5e7f10df3f2 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:21.806034 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce390c7-eda6-4096-abc6-d5e7f10df3f2 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:21.807723 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 136/270] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:21.821719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32229d41-fb0c-4128-bb62-dce94493913b None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:21.821719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32229d41-fb0c-4128-bb62-dce94493913b None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:21.821719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32229d41-fb0c-4128-bb62-dce94493913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:21.850060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-32229d41-fb0c-4128-bb62-dce94493913b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:21.851647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-32229d41-fb0c-4128-bb62-dce94493913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:21.931289 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 135/271] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:21.959342 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce390c7-eda6-4096-abc6-d5e7f10df3f2 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:22.084635 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce390c7-eda6-4096-abc6-d5e7f10df3f2 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:22.119328 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b25f5f-e925-4696-9ff3-2cc2ccb92955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:22.120504 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b25f5f-e925-4696-9ff3-2cc2ccb92955 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.120993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b25f5f-e925-4696-9ff3-2cc2ccb92955 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.121533 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b25f5f-e925-4696-9ff3-2cc2ccb92955 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.122969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b25f5f-e925-4696-9ff3-2cc2ccb92955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:22.124604 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b25f5f-e925-4696-9ff3-2cc2ccb92955 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:22.126791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 137/272] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:22.132706 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32229d41-fb0c-4128-bb62-dce94493913b None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:22.200875 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa10b85-668f-46ba-9408-d47e8e7cb318 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.201890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa10b85-668f-46ba-9408-d47e8e7cb318 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.202379 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa10b85-668f-46ba-9408-d47e8e7cb318 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.208189 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 138/273] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:22.211576 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae8b51-b7dd-4e83-b499-abb817160a37 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.212528 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae8b51-b7dd-4e83-b499-abb817160a37 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.213175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae8b51-b7dd-4e83-b499-abb817160a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.244697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-c2ae8b51-b7dd-4e83-b499-abb817160a37 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:22.247182 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-c2ae8b51-b7dd-4e83-b499-abb817160a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:22.308821 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32229d41-fb0c-4128-bb62-dce94493913b None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:22.346102 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 139/274] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:22.350825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f502a-69f9-4919-b6b5-94c9c76e9782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:22.351568 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f502a-69f9-4919-b6b5-94c9c76e9782 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.351870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f502a-69f9-4919-b6b5-94c9c76e9782 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.352162 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f502a-69f9-4919-b6b5-94c9c76e9782 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.353159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f502a-69f9-4919-b6b5-94c9c76e9782 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:22.355023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f502a-69f9-4919-b6b5-94c9c76e9782 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:22.374613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae8b51-b7dd-4e83-b499-abb817160a37 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:22.389093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 136/275] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:22 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:22.503323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae8b51-b7dd-4e83-b499-abb817160a37 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:22.543434 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14ca3c-782f-439f-bbbb-f465c138350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:22.544648 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14ca3c-782f-439f-bbbb-f465c138350b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.545215 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14ca3c-782f-439f-bbbb-f465c138350b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.545750 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14ca3c-782f-439f-bbbb-f465c138350b None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.558888 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14ca3c-782f-439f-bbbb-f465c138350b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-7a14ca3c-782f-439f-bbbb-f465c138350b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:22.567543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:22.569274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:22.570214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:22.573307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 140/276] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:22.581020 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f502a-69f9-4919-b6b5-94c9c76e9782 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:22.685492 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f502a-69f9-4919-b6b5-94c9c76e9782 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:22.726460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab7b9d-982b-45fd-b5dd-5f61e3af4615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:22.727160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7b9d-982b-45fd-b5dd-5f61e3af4615 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.727462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7b9d-982b-45fd-b5dd-5f61e3af4615 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.727748 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7b9d-982b-45fd-b5dd-5f61e3af4615 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.728541 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab7b9d-982b-45fd-b5dd-5f61e3af4615 None admin] RBAC: Adding query filter params (name=member) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:22.728861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab7b9d-982b-45fd-b5dd-5f61e3af4615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:22.730322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab7b9d-982b-45fd-b5dd-5f61e3af4615 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:22.740142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 137/277] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:22.745819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14ca3c-782f-439f-bbbb-f465c138350b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:22.851724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14ca3c-782f-439f-bbbb-f465c138350b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:22.890674 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2dced-f880-45c1-8943-77ccde026470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:22.891356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918742-5729-4f55-85b0-f81150ad5d5f None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.891654 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918742-5729-4f55-85b0-f81150ad5d5f None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.891952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918742-5729-4f55-85b0-f81150ad5d5f None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.892458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2dced-f880-45c1-8943-77ccde026470 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.892969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2dced-f880-45c1-8943-77ccde026470 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.893469 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2dced-f880-45c1-8943-77ccde026470 None admin] PATH_INFO: `/v3/users/9fe19f64b64a46c1ac8a422592da0434` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.893716 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 138/278] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:22.898800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccf37d-2e65-4756-80de-9a372551c9a3 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:22.899325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccf37d-2e65-4756-80de-9a372551c9a3 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:22.899833 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ccf37d-2e65-4756-80de-9a372551c9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:22.931535 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-c0ccf37d-2e65-4756-80de-9a372551c9a3 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:22.932997 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2dced-f880-45c1-8943-77ccde026470 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe19f64b64a46c1ac8a422592da0434)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:22.934796 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2dced-f880-45c1-8943-77ccde026470 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:22.937809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-c0ccf37d-2e65-4756-80de-9a372551c9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:22.938406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 141/279] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:22 2022] GET /identity/v3/users/9fe19f64b64a46c1ac8a422592da0434 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:22.947292 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2dced-f880-45c1-8943-77ccde026470 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.028508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 139/280] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:23.063439 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759acd-a0bf-41d9-a687-0a4c582a94a4 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.064069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759acd-a0bf-41d9-a687-0a4c582a94a4 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.065423 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759acd-a0bf-41d9-a687-0a4c582a94a4 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.067428 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 140/281] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:41:23.068480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32759acd-a0bf-41d9-a687-0a4c582a94a4 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.099893 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2dced-f880-45c1-8943-77ccde026470 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:23.138475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fba64-d7f8-422e-9189-389ad9e64fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:23.139574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fba64-d7f8-422e-9189-389ad9e64fe0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.140137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fba64-d7f8-422e-9189-389ad9e64fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.140628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fba64-d7f8-422e-9189-389ad9e64fe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.141718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fba64-d7f8-422e-9189-389ad9e64fe0 None admin] RBAC: Adding query filter params (role.id=22b9b0cd5e41452380f6aff61caa9f91, user.id=9fe19f64b64a46c1ac8a422592da0434, scope.system=all) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:23.142326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fba64-d7f8-422e-9189-389ad9e64fe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:23.143989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fba64-d7f8-422e-9189-389ad9e64fe0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:23.149707 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 142/282] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:41:22 2022] GET /identity/v3/role_assignments?user.id=9fe19f64b64a46c1ac8a422592da0434&scope.system=all&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:23.154594 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394f82-d1fb-47e4-8d7e-deacd942298b None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.155172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394f82-d1fb-47e4-8d7e-deacd942298b None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.155704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96394f82-d1fb-47e4-8d7e-deacd942298b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.179673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32759acd-a0bf-41d9-a687-0a4c582a94a4 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:23.189728 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-96394f82-d1fb-47e4-8d7e-deacd942298b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:23.190889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-96394f82-d1fb-47e4-8d7e-deacd942298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:23.259745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff5783-ff54-44de-8e73-a2a888bd65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:23.260580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff5783-ff54-44de-8e73-a2a888bd65b6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.260914 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff5783-ff54-44de-8e73-a2a888bd65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.261236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff5783-ff54-44de-8e73-a2a888bd65b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.274869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff5783-ff54-44de-8e73-a2a888bd65b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-1dff5783-ff54-44de-8e73-a2a888bd65b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:23.283962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:23.285118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:23.286087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 141/283] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:23.293050 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff5783-ff54-44de-8e73-a2a888bd65b6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.315522 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 143/284] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:23.358933 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96394f82-d1fb-47e4-8d7e-deacd942298b None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.463362 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff5783-ff54-44de-8e73-a2a888bd65b6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:23.491434 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96394f82-d1fb-47e4-8d7e-deacd942298b None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:23.525832 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:23.526574 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.526860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.527139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.527922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] RBAC: Adding query filter params (name=reader) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:23.528210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:23.529702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:23.530548 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67dac9-b019-4685-85d0-74594bb20e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:23.531619 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67dac9-b019-4685-85d0-74594bb20e6d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.532138 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67dac9-b019-4685-85d0-74594bb20e6d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.532632 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67dac9-b019-4685-85d0-74594bb20e6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.544826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67dac9-b019-4685-85d0-74594bb20e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:23.548651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 142/285] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-ed67dac9-b019-4685-85d0-74594bb20e6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:23.553500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:23.554552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:23.555625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:23.556358 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.558721 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 144/286] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:23.567673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67dac9-b019-4685-85d0-74594bb20e6d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.682947 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4922bfb-8800-4076-8613-ce689d881518 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:23.721756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d808d3-7f24-44d9-81a9-e1d6e4435a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:23.723738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d808d3-7f24-44d9-81a9-e1d6e4435a3c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.724277 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d808d3-7f24-44d9-81a9-e1d6e4435a3c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.724792 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d808d3-7f24-44d9-81a9-e1d6e4435a3c None admin] PATH_INFO: `/v3/users/d7a99989198e407089cff609e8765e92` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.726277 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d808d3-7f24-44d9-81a9-e1d6e4435a3c None admin] RBAC: Authorizing `identity:get_user(user_id=d7a99989198e407089cff609e8765e92)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:23.727971 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d808d3-7f24-44d9-81a9-e1d6e4435a3c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:23.730318 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 143/287] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:23 2022] GET /identity/v3/users/d7a99989198e407089cff609e8765e92 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:23.741909 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d808d3-7f24-44d9-81a9-e1d6e4435a3c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.776094 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67dac9-b019-4685-85d0-74594bb20e6d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:23.809672 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:23.810461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.810660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.810911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.811673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:23.812008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:23.813440 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:23.822566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 145/288] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:23.828190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.854988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d808d3-7f24-44d9-81a9-e1d6e4435a3c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:23.889794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59756f36-3a64-4c55-b330-e9be9f89e210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:23.891053 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59756f36-3a64-4c55-b330-e9be9f89e210 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.891560 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59756f36-3a64-4c55-b330-e9be9f89e210 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.892053 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59756f36-3a64-4c55-b330-e9be9f89e210 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.893522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59756f36-3a64-4c55-b330-e9be9f89e210 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:23.895224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59756f36-3a64-4c55-b330-e9be9f89e210 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:23.898643 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 144/289] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:23 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:23.904493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59756f36-3a64-4c55-b330-e9be9f89e210 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:23.941320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eea411-7bc2-4531-b25f-e322e65f8e38 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:23.980629 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9c438-ebcb-4586-9a61-43534036eb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:23.981528 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9c438-ebcb-4586-9a61-43534036eb01 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:23.981819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9c438-ebcb-4586-9a61-43534036eb01 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:23.982105 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9c438-ebcb-4586-9a61-43534036eb01 None admin] PATH_INFO: `/v3/users/8fd3998a8fca46c3b89a295d586dc35d` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:23.983352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9c438-ebcb-4586-9a61-43534036eb01 None admin] RBAC: Authorizing `identity:get_user(user_id=8fd3998a8fca46c3b89a295d586dc35d)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:23.993075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9c438-ebcb-4586-9a61-43534036eb01 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:23.993075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 146/290] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:23 2022] GET /identity/v3/users/8fd3998a8fca46c3b89a295d586dc35d => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:23.993075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9c438-ebcb-4586-9a61-43534036eb01 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:24.104850 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59756f36-3a64-4c55-b330-e9be9f89e210 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:24.164136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d3860-b775-413f-ae2c-d0817792bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:24.165062 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d3860-b775-413f-ae2c-d0817792bb35 None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:24.165448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d3860-b775-413f-ae2c-d0817792bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:24.165657 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d3860-b775-413f-ae2c-d0817792bb35 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de/users/d7a99989198e407089cff609e8765e92/roles/9178c520b46b41e48fc1659fe0129123` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:24.167675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d3860-b775-413f-ae2c-d0817792bb35 None admin] RBAC: Authorizing `identity:create_grant(project_id=1322e64cad064e79a31ef6484516a1de, user_id=d7a99989198e407089cff609e8765e92, role_id=9178c520b46b41e48fc1659fe0129123)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:24.169277 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d3860-b775-413f-ae2c-d0817792bb35 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:24.172789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9c438-ebcb-4586-9a61-43534036eb01 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:24.194577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 145/291] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:41:23 2022] PUT /identity/v3/projects/1322e64cad064e79a31ef6484516a1de/users/d7a99989198e407089cff609e8765e92/roles/9178c520b46b41e48fc1659fe0129123 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:41:24.225607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56013a4-2508-43a0-8cef-812a74922f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:24.226284 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56013a4-2508-43a0-8cef-812a74922f58 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:24.226571 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56013a4-2508-43a0-8cef-812a74922f58 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:24.226850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56013a4-2508-43a0-8cef-812a74922f58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:24.327779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56013a4-2508-43a0-8cef-812a74922f58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:24.330092 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56013a4-2508-43a0-8cef-812a74922f58 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:24.451122 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 147/292] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:24.456577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d3860-b775-413f-ae2c-d0817792bb35 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:24.578952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d3860-b775-413f-ae2c-d0817792bb35 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:24.616130 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a316b-8e16-46ae-b8f6-290bb2e034d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:24.617020 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a316b-8e16-46ae-b8f6-290bb2e034d7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:24.617340 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a316b-8e16-46ae-b8f6-290bb2e034d7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:24.617624 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a316b-8e16-46ae-b8f6-290bb2e034d7 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:24.618985 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a316b-8e16-46ae-b8f6-290bb2e034d7 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:24.620459 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a316b-8e16-46ae-b8f6-290bb2e034d7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:24.622602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 146/293] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:24 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:24.628025 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56013a4-2508-43a0-8cef-812a74922f58 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:24.730345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56013a4-2508-43a0-8cef-812a74922f58 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:24.780260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7c626-9eca-4dbd-8029-a53b6d8f4c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:24.780938 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c626-9eca-4dbd-8029-a53b6d8f4c26 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:24.781263 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c626-9eca-4dbd-8029-a53b6d8f4c26 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:24.781549 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c626-9eca-4dbd-8029-a53b6d8f4c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:24.782412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7c626-9eca-4dbd-8029-a53b6d8f4c26 None admin] RBAC: Adding query filter params (role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=1322e64cad064e79a31ef6484516a1de, user.id=8fd3998a8fca46c3b89a295d586dc35d) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:24.782739 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7c626-9eca-4dbd-8029-a53b6d8f4c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:24.784196 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7c626-9eca-4dbd-8029-a53b6d8f4c26 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:24.793727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 148/294] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:24 2022] GET /identity/v3/role_assignments?user.id=8fd3998a8fca46c3b89a295d586dc35d&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:25.192349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ccaf0-4750-48f3-ab27-a292094c6194 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.192552 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ccaf0-4750-48f3-ab27-a292094c6194 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.192865 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ccaf0-4750-48f3-ab27-a292094c6194 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.194625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 147/295] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:25.200146 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c3bb-072c-4d2f-9233-8a6c5906b137 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.200452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c3bb-072c-4d2f-9233-8a6c5906b137 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.200741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c3bb-072c-4d2f-9233-8a6c5906b137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.218400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-2137c3bb-072c-4d2f-9233-8a6c5906b137 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:25.219615 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-2137c3bb-072c-4d2f-9233-8a6c5906b137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:25.297150 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 149/296] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:25.314380 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ccaf0-4750-48f3-ab27-a292094c6194 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:25.332735 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39359aa-a472-403b-bc1f-dd24e000bcbf None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.333815 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39359aa-a472-403b-bc1f-dd24e000bcbf None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.333935 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39359aa-a472-403b-bc1f-dd24e000bcbf None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.336338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 150/297] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:41:25.348024 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4bd2-a7e6-499f-a34a-b6f93fa4509f None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.348328 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4bd2-a7e6-499f-a34a-b6f93fa4509f None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.348599 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db4bd2-a7e6-499f-a34a-b6f93fa4509f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.369480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-e8db4bd2-a7e6-499f-a34a-b6f93fa4509f None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:25.370696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-e8db4bd2-a7e6-499f-a34a-b6f93fa4509f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:25.458192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 151/298] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:25.460749 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ccaf0-4750-48f3-ab27-a292094c6194 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:25.484295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db4bd2-a7e6-499f-a34a-b6f93fa4509f None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:25.503840 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b607c31-bc7c-4c3f-8c92-da237d505446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:25.505119 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b607c31-bc7c-4c3f-8c92-da237d505446 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.506145 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b607c31-bc7c-4c3f-8c92-da237d505446 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.506639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b607c31-bc7c-4c3f-8c92-da237d505446 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.517793 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b607c31-bc7c-4c3f-8c92-da237d505446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-0b607c31-bc7c-4c3f-8c92-da237d505446 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:25.525895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:25.526924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:25.527876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:25.530674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 148/299] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:25.542817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b607c31-bc7c-4c3f-8c92-da237d505446 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:25.609097 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db4bd2-a7e6-499f-a34a-b6f93fa4509f None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:25.653266 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecffc3c-4f52-4aa8-8c8e-76a38076c10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:25.654568 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecffc3c-4f52-4aa8-8c8e-76a38076c10b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.654979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecffc3c-4f52-4aa8-8c8e-76a38076c10b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.655269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecffc3c-4f52-4aa8-8c8e-76a38076c10b None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.656509 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecffc3c-4f52-4aa8-8c8e-76a38076c10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:25.657106 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b607c31-bc7c-4c3f-8c92-da237d505446 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:25.658145 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecffc3c-4f52-4aa8-8c8e-76a38076c10b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:25.664013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 152/300] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:25.670134 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecffc3c-4f52-4aa8-8c8e-76a38076c10b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:25.712183 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:25.713059 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.713424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.714561 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.715480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] RBAC: Adding query filter params (name=reader) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:25.715870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:25.717363 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:25.728367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 149/301] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:25.732966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:25.803391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecffc3c-4f52-4aa8-8c8e-76a38076c10b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:25.843746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14116db5-218e-4c6d-9e7b-cd48eafd0a30 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:25.843989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8c29e-9a22-4997-bdd7-90c97b16e606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:25.844509 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8c29e-9a22-4997-bdd7-90c97b16e606 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.844800 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8c29e-9a22-4997-bdd7-90c97b16e606 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.845113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8c29e-9a22-4997-bdd7-90c97b16e606 None admin] PATH_INFO: `/v3/groups` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.845821 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8c29e-9a22-4997-bdd7-90c97b16e606 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:25.848545 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8c29e-9a22-4997-bdd7-90c97b16e606 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:25.873069 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 153/302] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 9 15:41:25 2022] POST /identity/v3/groups => generated 229 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:25.890187 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43525108-c9cd-4d81-93a3-cb0a37a19be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:25.891093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525108-c9cd-4d81-93a3-cb0a37a19be1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.891382 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525108-c9cd-4d81-93a3-cb0a37a19be1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.891660 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525108-c9cd-4d81-93a3-cb0a37a19be1 None admin] PATH_INFO: `/v3/users/4fb1abe2c3324055977c2e70f9e3bbd3` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.919576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43525108-c9cd-4d81-93a3-cb0a37a19be1 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb1abe2c3324055977c2e70f9e3bbd3)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:25.921215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43525108-c9cd-4d81-93a3-cb0a37a19be1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:25.923326 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 150/303] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:25 2022] GET /identity/v3/users/4fb1abe2c3324055977c2e70f9e3bbd3 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:25.930605 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8c29e-9a22-4997-bdd7-90c97b16e606 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:25.951769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0616d2-eff1-4d80-af0f-e241372e4870 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.952166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0616d2-eff1-4d80-af0f-e241372e4870 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.952343 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0616d2-eff1-4d80-af0f-e241372e4870 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.954708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 151/304] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:25.962444 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9265652-459a-4808-9c99-e2278dd51682 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:25.962644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9265652-459a-4808-9c99-e2278dd51682 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:25.963012 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9265652-459a-4808-9c99-e2278dd51682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:25.988614 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-a9265652-459a-4808-9c99-e2278dd51682 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:25.990919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-a9265652-459a-4808-9c99-e2278dd51682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:26.052652 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8c29e-9a22-4997-bdd7-90c97b16e606 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:26.082222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 152/305] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:26.095612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:26.096277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:26.096564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:26.096848 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:26.097731 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] RBAC: Adding query filter params (role.id=9178c520b46b41e48fc1659fe0129123, user.id=4fb1abe2c3324055977c2e70f9e3bbd3, scope.system=all) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:26.098308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:26.100268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:26.105889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9265652-459a-4808-9c99-e2278dd51682 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:26.109505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 154/306] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:41:25 2022] GET /identity/v3/role_assignments?user.id=4fb1abe2c3324055977c2e70f9e3bbd3&scope.system=all&role.id=9178c520b46b41e48fc1659fe0129123 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:26.228440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9265652-459a-4808-9c99-e2278dd51682 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:26.265729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5051ccd-956a-43de-a806-ec74f7cd77d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:26.266560 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5051ccd-956a-43de-a806-ec74f7cd77d0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:26.266851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5051ccd-956a-43de-a806-ec74f7cd77d0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:26.267135 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5051ccd-956a-43de-a806-ec74f7cd77d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:26.279391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5051ccd-956a-43de-a806-ec74f7cd77d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-e5051ccd-956a-43de-a806-ec74f7cd77d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:26.288114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:26.289261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:26.290221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 153/307] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:26.297544 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:26.403261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf68eb9-cc62-4e18-adbe-af0add172fe8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:26.437315 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:26.438003 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:26.438293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:26.438578 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:26.439354 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:26.439670 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:26.441074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:26.450036 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 155/308] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:26.459628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5051ccd-956a-43de-a806-ec74f7cd77d0 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:26.561305 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5051ccd-956a-43de-a806-ec74f7cd77d0 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:26.596949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dada6a4-d130-4273-b71a-4d59880e1d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:26.598147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dada6a4-d130-4273-b71a-4d59880e1d5b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:26.598445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dada6a4-d130-4273-b71a-4d59880e1d5b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:26.598728 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dada6a4-d130-4273-b71a-4d59880e1d5b None admin] PATH_INFO: `/v3/users/9fe19f64b64a46c1ac8a422592da0434` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:26.600508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dada6a4-d130-4273-b71a-4d59880e1d5b None admin] RBAC: Authorizing `identity:get_user(user_id=9fe19f64b64a46c1ac8a422592da0434)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:26.602012 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dada6a4-d130-4273-b71a-4d59880e1d5b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:26.603973 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 154/309] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:26 2022] GET /identity/v3/users/9fe19f64b64a46c1ac8a422592da0434 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:26.609329 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:26.715572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2c8b7-01d5-43d6-a0b6-112d66a20192 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:26.753780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e993c6-21a4-44e6-8e87-2fa78779e28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:26.754656 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e993c6-21a4-44e6-8e87-2fa78779e28e None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:26.754943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e993c6-21a4-44e6-8e87-2fa78779e28e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:26.755758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e993c6-21a4-44e6-8e87-2fa78779e28e None admin] PATH_INFO: `/v3/system/users/9fe19f64b64a46c1ac8a422592da0434/roles/22b9b0cd5e41452380f6aff61caa9f91` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:26.757314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e993c6-21a4-44e6-8e87-2fa78779e28e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9fe19f64b64a46c1ac8a422592da0434, role_id=22b9b0cd5e41452380f6aff61caa9f91)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:26.758698 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e993c6-21a4-44e6-8e87-2fa78779e28e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:26.769246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 156/310] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 9 15:41:26 2022] PUT /identity/v3/system/users/9fe19f64b64a46c1ac8a422592da0434/roles/22b9b0cd5e41452380f6aff61caa9f91 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:26.915269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae5629-e497-48e6-9e40-a2ae3edb562e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:26.915671 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae5629-e497-48e6-9e40-a2ae3edb562e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:26.915857 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae5629-e497-48e6-9e40-a2ae3edb562e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:26.917598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 155/311] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:26.923605 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bced7-f3ba-4496-994f-666ceeb741f9 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:26.923913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bced7-f3ba-4496-994f-666ceeb741f9 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:26.924181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bced7-f3ba-4496-994f-666ceeb741f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:26.942126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-298bced7-f3ba-4496-994f-666ceeb741f9 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:26.944343 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-298bced7-f3ba-4496-994f-666ceeb741f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:27.015743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 157/312] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:27.039147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae5629-e497-48e6-9e40-a2ae3edb562e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:27.143270 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae5629-e497-48e6-9e40-a2ae3edb562e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:27.179378 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b4a7-2ddb-4cb5-a5e7-b1af1e572e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:27.180123 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b4a7-2ddb-4cb5-a5e7-b1af1e572e99 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:27.180399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b4a7-2ddb-4cb5-a5e7-b1af1e572e99 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:27.180683 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b4a7-2ddb-4cb5-a5e7-b1af1e572e99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:27.190946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5b4a7-2ddb-4cb5-a5e7-b1af1e572e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-72d5b4a7-2ddb-4cb5-a5e7-b1af1e572e99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:27.199207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:27.200232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:27.201621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 156/313] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:27.208561 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bced7-f3ba-4496-994f-666ceeb741f9 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:27.310970 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bced7-f3ba-4496-994f-666ceeb741f9 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:27.347529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:27.348486 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:27.348813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:27.349129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:27.349904 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] RBAC: Adding query filter params (name=reader) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:27.350221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:27.351611 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:27.360275 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 158/314] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:27.366530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b4a7-2ddb-4cb5-a5e7-b1af1e572e99 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:27.469227 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b4a7-2ddb-4cb5-a5e7-b1af1e572e99 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:27.508110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993ca41-217e-41cb-8202-6ed971afa885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:27.509028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ca41-217e-41cb-8202-6ed971afa885 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:27.509369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ca41-217e-41cb-8202-6ed971afa885 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:27.509663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ca41-217e-41cb-8202-6ed971afa885 None admin] PATH_INFO: `/v3/users/d7a99989198e407089cff609e8765e92` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:27.511084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993ca41-217e-41cb-8202-6ed971afa885 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a99989198e407089cff609e8765e92)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:27.512622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993ca41-217e-41cb-8202-6ed971afa885 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:27.514723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 157/315] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:27 2022] GET /identity/v3/users/d7a99989198e407089cff609e8765e92 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:27.520281 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:27.623868 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140c227-1967-4e9f-96e0-9cee0c0f047d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:27.658132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8675e-84e5-4c59-923a-2ab181bd401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:27.658795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8675e-84e5-4c59-923a-2ab181bd401c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:27.659085 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8675e-84e5-4c59-923a-2ab181bd401c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:27.659369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8675e-84e5-4c59-923a-2ab181bd401c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:27.726761 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8675e-84e5-4c59-923a-2ab181bd401c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:27.728141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8675e-84e5-4c59-923a-2ab181bd401c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:27.827631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 159/316] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:27.833611 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993ca41-217e-41cb-8202-6ed971afa885 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:27.940434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993ca41-217e-41cb-8202-6ed971afa885 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:27.975799 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fc543-59c1-4044-9bd5-59290b6597a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:27.976684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc543-59c1-4044-9bd5-59290b6597a3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:27.977054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc543-59c1-4044-9bd5-59290b6597a3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:27.977296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fc543-59c1-4044-9bd5-59290b6597a3 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:27.978533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fc543-59c1-4044-9bd5-59290b6597a3 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:27.979997 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fc543-59c1-4044-9bd5-59290b6597a3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:27.982071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 158/317] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:27 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:27.987957 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8675e-84e5-4c59-923a-2ab181bd401c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:28.098282 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8675e-84e5-4c59-923a-2ab181bd401c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:28.133346 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d769ee-f326-4779-8d13-bdab67a17506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:28.134016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d769ee-f326-4779-8d13-bdab67a17506 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.134309 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d769ee-f326-4779-8d13-bdab67a17506 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.134594 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d769ee-f326-4779-8d13-bdab67a17506 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.135446 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d769ee-f326-4779-8d13-bdab67a17506 None admin] RBAC: Adding query filter params (role.id=9178c520b46b41e48fc1659fe0129123, scope.project.id=1322e64cad064e79a31ef6484516a1de, user.id=d7a99989198e407089cff609e8765e92) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:28.135768 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d769ee-f326-4779-8d13-bdab67a17506 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:28.137240 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d769ee-f326-4779-8d13-bdab67a17506 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:28.143892 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 160/318] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:27 2022] GET /identity/v3/role_assignments?user.id=d7a99989198e407089cff609e8765e92&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=9178c520b46b41e48fc1659fe0129123 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:28.356140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb494d42-b1de-4c5c-8706-39ee69735fb3 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.356525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb494d42-b1de-4c5c-8706-39ee69735fb3 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.356639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb494d42-b1de-4c5c-8706-39ee69735fb3 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.358396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 159/319] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:28.366217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7600aa-b560-411f-b27a-03a7b95aaf60 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.367046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7600aa-b560-411f-b27a-03a7b95aaf60 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.367830 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7600aa-b560-411f-b27a-03a7b95aaf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.387458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-7a7600aa-b560-411f-b27a-03a7b95aaf60 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:28.389113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-7a7600aa-b560-411f-b27a-03a7b95aaf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:28.460232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 161/320] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:28.484118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb494d42-b1de-4c5c-8706-39ee69735fb3 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:28.523342 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574ec1-a969-4ef3-8159-77f82f344073 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.523898 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574ec1-a969-4ef3-8159-77f82f344073 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.523898 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574ec1-a969-4ef3-8159-77f82f344073 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.525469 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 162/321] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:41:28.530252 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531442b1-9b64-4c3e-9cc9-5f2f32b3e580 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.530545 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531442b1-9b64-4c3e-9cc9-5f2f32b3e580 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.530831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531442b1-9b64-4c3e-9cc9-5f2f32b3e580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.548034 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-531442b1-9b64-4c3e-9cc9-5f2f32b3e580 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:28.549254 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-531442b1-9b64-4c3e-9cc9-5f2f32b3e580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:28.599397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb494d42-b1de-4c5c-8706-39ee69735fb3 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:28.615799 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 163/322] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:28.634882 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fd662-2161-4ebe-a743-4fcd27dcda46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:28.635681 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fd662-2161-4ebe-a743-4fcd27dcda46 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.635967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fd662-2161-4ebe-a743-4fcd27dcda46 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.636244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fd662-2161-4ebe-a743-4fcd27dcda46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.637272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531442b1-9b64-4c3e-9cc9-5f2f32b3e580 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:28.638606 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7fd662-2161-4ebe-a743-4fcd27dcda46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:28.641425 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7fd662-2161-4ebe-a743-4fcd27dcda46 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:28.643334 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 160/323] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:28.649339 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fd662-2161-4ebe-a743-4fcd27dcda46 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:28.736596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531442b1-9b64-4c3e-9cc9-5f2f32b3e580 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:28.749098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fd662-2161-4ebe-a743-4fcd27dcda46 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:28.770005 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00db217-fe0e-4cd6-8def-96df933286d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:28.771337 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00db217-fe0e-4cd6-8def-96df933286d7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.771633 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00db217-fe0e-4cd6-8def-96df933286d7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.771912 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00db217-fe0e-4cd6-8def-96df933286d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.781993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00db217-fe0e-4cd6-8def-96df933286d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:28.782848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91ea0f-4b00-4411-ab43-c626886ec906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:28.783521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91ea0f-4b00-4411-ab43-c626886ec906 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.783815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91ea0f-4b00-4411-ab43-c626886ec906 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.784101 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91ea0f-4b00-4411-ab43-c626886ec906 None admin] PATH_INFO: `/v3/groups` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.784797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91ea0f-4b00-4411-ab43-c626886ec906 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:28.786843 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91ea0f-4b00-4411-ab43-c626886ec906 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-a00db217-fe0e-4cd6-8def-96df933286d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:28.790387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:28.791542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:28.792587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 164/324] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:28.797483 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00db217-fe0e-4cd6-8def-96df933286d7 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:28.813435 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 161/325] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 9 15:41:28 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:41:28.912149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00db217-fe0e-4cd6-8def-96df933286d7 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:28.948046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67125b-dc9c-4880-adac-a6768fee3123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:28.948746 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67125b-dc9c-4880-adac-a6768fee3123 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:28.949062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67125b-dc9c-4880-adac-a6768fee3123 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:28.949352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67125b-dc9c-4880-adac-a6768fee3123 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:28.950216 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67125b-dc9c-4880-adac-a6768fee3123 None admin] RBAC: Adding query filter params (name=reader) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:28.950539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67125b-dc9c-4880-adac-a6768fee3123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:28.951972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67125b-dc9c-4880-adac-a6768fee3123 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:28.962272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 165/326] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:28.967972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91ea0f-4b00-4411-ab43-c626886ec906 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:29.036447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e19c32-2588-4637-95ab-1209585c955e None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:29.036744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e19c32-2588-4637-95ab-1209585c955e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:29.037265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e19c32-2588-4637-95ab-1209585c955e None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:29.038729 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 166/327] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:41:29.044925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091c236-d94f-4bbf-acf3-71ae070d2a42 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:29.045249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091c236-d94f-4bbf-acf3-71ae070d2a42 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:29.045539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091c236-d94f-4bbf-acf3-71ae070d2a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:29.063669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-8091c236-d94f-4bbf-acf3-71ae070d2a42 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:29.064907 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-8091c236-d94f-4bbf-acf3-71ae070d2a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:29.071845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91ea0f-4b00-4411-ab43-c626886ec906 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:29.112419 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468edcf3-56b0-455d-a459-8d3453cee750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:29.113493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468edcf3-56b0-455d-a459-8d3453cee750 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:29.113752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468edcf3-56b0-455d-a459-8d3453cee750 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:29.114040 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468edcf3-56b0-455d-a459-8d3453cee750 None admin] PATH_INFO: `/v3/users/4fb1abe2c3324055977c2e70f9e3bbd3` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:29.115826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468edcf3-56b0-455d-a459-8d3453cee750 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb1abe2c3324055977c2e70f9e3bbd3)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:29.117694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468edcf3-56b0-455d-a459-8d3453cee750 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:29.120026 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 162/328] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:28 2022] GET /identity/v3/users/4fb1abe2c3324055977c2e70f9e3bbd3 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:29.125771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468edcf3-56b0-455d-a459-8d3453cee750 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:29.154505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 167/329] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:29.177767 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091c236-d94f-4bbf-acf3-71ae070d2a42 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:29.254671 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468edcf3-56b0-455d-a459-8d3453cee750 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:29.291343 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0f8c2-bdc6-4281-9285-f125ef2efe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:29.292238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f8c2-bdc6-4281-9285-f125ef2efe93 None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:29.292530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f8c2-bdc6-4281-9285-f125ef2efe93 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:29.292820 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0f8c2-bdc6-4281-9285-f125ef2efe93 None admin] PATH_INFO: `/v3/system/users/4fb1abe2c3324055977c2e70f9e3bbd3/roles/9178c520b46b41e48fc1659fe0129123` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:29.294402 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0f8c2-bdc6-4281-9285-f125ef2efe93 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4fb1abe2c3324055977c2e70f9e3bbd3, role_id=9178c520b46b41e48fc1659fe0129123)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:29.294533 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091c236-d94f-4bbf-acf3-71ae070d2a42 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:29.295802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0f8c2-bdc6-4281-9285-f125ef2efe93 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:29.307474 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 163/330] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 9 15:41:29 2022] PUT /identity/v3/system/users/4fb1abe2c3324055977c2e70f9e3bbd3/roles/9178c520b46b41e48fc1659fe0129123 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:41:29.333973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389350d1-034c-4a2a-801c-6ea3bc7f1caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:29.334801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389350d1-034c-4a2a-801c-6ea3bc7f1caa None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:29.335090 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389350d1-034c-4a2a-801c-6ea3bc7f1caa None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:29.335368 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389350d1-034c-4a2a-801c-6ea3bc7f1caa None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:29.348319 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389350d1-034c-4a2a-801c-6ea3bc7f1caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-389350d1-034c-4a2a-801c-6ea3bc7f1caa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:29.356494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:29.357646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:29.358608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 168/331] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:29.366415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0f8c2-bdc6-4281-9285-f125ef2efe93 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:29.487390 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0f8c2-bdc6-4281-9285-f125ef2efe93 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:29.526431 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:29.527171 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:29.527467 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:29.527753 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:29.528531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] RBAC: Adding query filter params (name=member) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:29.528848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:29.530291 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:29.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 164/332] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:29.544625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389350d1-034c-4a2a-801c-6ea3bc7f1caa None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:29.658109 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389350d1-034c-4a2a-801c-6ea3bc7f1caa None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:29.694812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0da5e-c14d-42e0-ab15-b1afab2ca95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:29.695696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0da5e-c14d-42e0-ab15-b1afab2ca95d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:29.695980 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0da5e-c14d-42e0-ab15-b1afab2ca95d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:29.696257 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0da5e-c14d-42e0-ab15-b1afab2ca95d None admin] PATH_INFO: `/v3/users/9fe19f64b64a46c1ac8a422592da0434` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:29.697534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0da5e-c14d-42e0-ab15-b1afab2ca95d None admin] RBAC: Authorizing `identity:get_user(user_id=9fe19f64b64a46c1ac8a422592da0434)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:29.699087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0da5e-c14d-42e0-ab15-b1afab2ca95d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:29.700775 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 169/333] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:29 2022] GET /identity/v3/users/9fe19f64b64a46c1ac8a422592da0434 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:29.706685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:29.842672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1057c7-94ca-41a9-b4a1-30f9f302d271 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:29.893922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812d2e6-f1cc-47df-92ff-7a64a190af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:29.894597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d2e6-f1cc-47df-92ff-7a64a190af26 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:29.894892 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d2e6-f1cc-47df-92ff-7a64a190af26 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:29.895177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812d2e6-f1cc-47df-92ff-7a64a190af26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:29.896097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812d2e6-f1cc-47df-92ff-7a64a190af26 None admin] RBAC: Adding query filter params (role.id=22b9b0cd5e41452380f6aff61caa9f91, user.id=9fe19f64b64a46c1ac8a422592da0434, scope.system=all) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:29.896420 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812d2e6-f1cc-47df-92ff-7a64a190af26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:29.898007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812d2e6-f1cc-47df-92ff-7a64a190af26 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:29.927498 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 165/334] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:41:29 2022] GET /identity/v3/role_assignments?user.id=9fe19f64b64a46c1ac8a422592da0434&scope.system=all&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:31.821924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242128-dbc5-488c-9027-b7ca41a9b21c None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:31.822503 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242128-dbc5-488c-9027-b7ca41a9b21c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:31.822503 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4242128-dbc5-488c-9027-b7ca41a9b21c None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:31.823854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 170/335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:31.829498 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1f080-068b-4ecf-8182-0e4c2f12ae2b None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:31.829956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1f080-068b-4ecf-8182-0e4c2f12ae2b None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:31.830068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1f080-068b-4ecf-8182-0e4c2f12ae2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:31.850488 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-11c1f080-068b-4ecf-8182-0e4c2f12ae2b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:31.851701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-11c1f080-068b-4ecf-8182-0e4c2f12ae2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:31.868880 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d1b8d-72c2-4237-ac49-2d58b9565f86 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:31.869239 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d1b8d-72c2-4237-ac49-2d58b9565f86 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:31.869515 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d1b8d-72c2-4237-ac49-2d58b9565f86 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:31.871132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 171/336] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:31.877552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8b4f-5aa2-4710-a820-1b9acdfd2026 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:31.878140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8b4f-5aa2-4710-a820-1b9acdfd2026 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:31.878639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8b4f-5aa2-4710-a820-1b9acdfd2026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:31.908052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-346f8b4f-5aa2-4710-a820-1b9acdfd2026 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:31.908921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-346f8b4f-5aa2-4710-a820-1b9acdfd2026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:31.970676 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 166/337] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:32.004774 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 172/338] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:32.007052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1f080-068b-4ecf-8182-0e4c2f12ae2b None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:32.045562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8b4f-5aa2-4710-a820-1b9acdfd2026 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:32.120574 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1f080-068b-4ecf-8182-0e4c2f12ae2b None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:32.160656 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be64abf-009d-41e5-8894-dd9905e8ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:32.162069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be64abf-009d-41e5-8894-dd9905e8ca1e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.162357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be64abf-009d-41e5-8894-dd9905e8ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.162639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be64abf-009d-41e5-8894-dd9905e8ca1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.175250 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be64abf-009d-41e5-8894-dd9905e8ca1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9be64abf-009d-41e5-8894-dd9905e8ca1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:32.184896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:32.186087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:32.187632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 167/339] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:32.193535 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be64abf-009d-41e5-8894-dd9905e8ca1e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:32.281015 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8b4f-5aa2-4710-a820-1b9acdfd2026 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:32.314773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be64abf-009d-41e5-8894-dd9905e8ca1e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:32.353862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:32.354552 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.354839 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.355178 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.355950 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:32.356264 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:32.357715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:32.358367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da735975-fd52-4845-94eb-aa20363c1c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:32.359176 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da735975-fd52-4845-94eb-aa20363c1c45 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.359465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da735975-fd52-4845-94eb-aa20363c1c45 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.359747 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da735975-fd52-4845-94eb-aa20363c1c45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.366838 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 168/340] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:32.373076 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:32.385987 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da735975-fd52-4845-94eb-aa20363c1c45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-da735975-fd52-4845-94eb-aa20363c1c45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:32.394013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:32.397140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:32.399314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:32.405140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 173/341] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:32.410158 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da735975-fd52-4845-94eb-aa20363c1c45 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:32.491765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1796e-2cf2-44f2-9df2-eea43daa2efe None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:32.550509 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa6862-8ada-4fc5-a413-d05f6062932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:32.551590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6862-8ada-4fc5-a413-d05f6062932a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.552160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6862-8ada-4fc5-a413-d05f6062932a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.552671 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6862-8ada-4fc5-a413-d05f6062932a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.585678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da735975-fd52-4845-94eb-aa20363c1c45 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:32.624498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd4b5d-202b-4991-9065-31729a199ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:32.625136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd4b5d-202b-4991-9065-31729a199ee2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.625431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd4b5d-202b-4991-9065-31729a199ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.625717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd4b5d-202b-4991-9065-31729a199ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.626492 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd4b5d-202b-4991-9065-31729a199ee2 None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:32.626808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd4b5d-202b-4991-9065-31729a199ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:32.630507 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd4b5d-202b-4991-9065-31729a199ee2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:32.639107 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 174/342] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:32.640436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac3e6c-9805-44c2-ad24-e07d8a1e5eea None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.640728 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac3e6c-9805-44c2-ad24-e07d8a1e5eea None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.641012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac3e6c-9805-44c2-ad24-e07d8a1e5eea None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.642756 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 175/343] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:32.649660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3738edb-1b34-485b-ae05-188e31a61a34 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.649959 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3738edb-1b34-485b-ae05-188e31a61a34 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.650248 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3738edb-1b34-485b-ae05-188e31a61a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.679524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-a3738edb-1b34-485b-ae05-188e31a61a34 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:32.680550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa6862-8ada-4fc5-a413-d05f6062932a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:32.680839 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-a3738edb-1b34-485b-ae05-188e31a61a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:32.682583 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa6862-8ada-4fc5-a413-d05f6062932a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:32.754554 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 176/344] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:32.755935 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3738edb-1b34-485b-ae05-188e31a61a34 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:32.788826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 169/345] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:41:32.790680 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367688-2c38-4dd3-9e42-39fccdaf80d2 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.791222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367688-2c38-4dd3-9e42-39fccdaf80d2 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.791778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367688-2c38-4dd3-9e42-39fccdaf80d2 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.793610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 170/346] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:32.798068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367688-2c38-4dd3-9e42-39fccdaf80d2 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:32.882874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3738edb-1b34-485b-ae05-188e31a61a34 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:32.920877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe15ed-aeac-4116-bbea-aafe467a5769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:32.920877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe15ed-aeac-4116-bbea-aafe467a5769 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.920877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe15ed-aeac-4116-bbea-aafe467a5769 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.921246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe15ed-aeac-4116-bbea-aafe467a5769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.923061 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367688-2c38-4dd3-9e42-39fccdaf80d2 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:32.960588 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7985d-9031-4082-9377-9957c46a6220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:32.961972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7985d-9031-4082-9377-9957c46a6220 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.962539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7985d-9031-4082-9377-9957c46a6220 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.963103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7985d-9031-4082-9377-9957c46a6220 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.965210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7985d-9031-4082-9377-9957c46a6220 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:32.967056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7985d-9031-4082-9377-9957c46a6220 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:32.973196 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 171/347] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:32 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:32.974848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac8288-fbfa-4180-9612-0933e78aaee8 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:32.975374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac8288-fbfa-4180-9612-0933e78aaee8 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:32.975893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac8288-fbfa-4180-9612-0933e78aaee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:32.994280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe15ed-aeac-4116-bbea-aafe467a5769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:32.995750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-1aac8288-fbfa-4180-9612-0933e78aaee8 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:32.995899 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe15ed-aeac-4116-bbea-aafe467a5769 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:32.997373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-1aac8288-fbfa-4180-9612-0933e78aaee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:33.068935 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 172/348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:33.070379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac8288-fbfa-4180-9612-0933e78aaee8 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.100654 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 177/349] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:33.101482 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe15ed-aeac-4116-bbea-aafe467a5769 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.174086 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac8288-fbfa-4180-9612-0933e78aaee8 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.204440 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe15ed-aeac-4116-bbea-aafe467a5769 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.210520 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74fea9-640d-4876-b3e9-dae41d313703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.211639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74fea9-640d-4876-b3e9-dae41d313703 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.212156 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74fea9-640d-4876-b3e9-dae41d313703 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.212764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74fea9-640d-4876-b3e9-dae41d313703 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.224208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74fea9-640d-4876-b3e9-dae41d313703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9a74fea9-640d-4876-b3e9-dae41d313703 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:33.232623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:33.233712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 09 15:41:33.234665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:33.237264 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 173/350] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:33.238451 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74fea9-640d-4876-b3e9-dae41d313703 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.238854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb248-647b-4277-8cf1-58a5d9a7e38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.240281 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb248-647b-4277-8cf1-58a5d9a7e38d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.240566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb248-647b-4277-8cf1-58a5d9a7e38d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.240844 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb248-647b-4277-8cf1-58a5d9a7e38d None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.247125 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaeb248-647b-4277-8cf1-58a5d9a7e38d None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:33.248584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaeb248-647b-4277-8cf1-58a5d9a7e38d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:33.250293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 178/351] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:33.251056 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb248-647b-4277-8cf1-58a5d9a7e38d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.344269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74fea9-640d-4876-b3e9-dae41d313703 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.353660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb248-647b-4277-8cf1-58a5d9a7e38d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.380552 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d793ff-8ddb-484e-949e-a5135022eb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.381845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d793ff-8ddb-484e-949e-a5135022eb57 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.382369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d793ff-8ddb-484e-949e-a5135022eb57 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.382894 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d793ff-8ddb-484e-949e-a5135022eb57 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.384383 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d793ff-8ddb-484e-949e-a5135022eb57 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:33.386091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d793ff-8ddb-484e-949e-a5135022eb57 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:33.388421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9f87-78b1-4597-aa26-143eb1bd63e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.388745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 174/352] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:33.389214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9f87-78b1-4597-aa26-143eb1bd63e2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.389510 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9f87-78b1-4597-aa26-143eb1bd63e2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.389809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f9f87-78b1-4597-aa26-143eb1bd63e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.390642 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d793ff-8ddb-484e-949e-a5135022eb57 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.400681 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f9f87-78b1-4597-aa26-143eb1bd63e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-5d5f9f87-78b1-4597-aa26-143eb1bd63e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:33.408562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:33.409700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 179/353] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:33.411780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9f87-78b1-4597-aa26-143eb1bd63e2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.511881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d793ff-8ddb-484e-949e-a5135022eb57 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.523452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f9f87-78b1-4597-aa26-143eb1bd63e2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.547803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.548847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.549375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.549870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.550837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:33.551372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:33.553663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:33.558809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.559826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.559826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.560046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.560880 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] RBAC: Adding query filter params (group.id=8537bbbb4fcf49d0a56e6eaa390573ce, role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:33.561235 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:33.562696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:33.564280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 175/354] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:33.565543 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.570145 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 180/355] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/role_assignments?group.id=8537bbbb4fcf49d0a56e6eaa390573ce&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=044a2d93445743af91a96573d39f21eb => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:33.570606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.681094 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e71e7-256e-4e9c-b9e5-22bacd303c9c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.681873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44ffb9-2aee-4cc1-ad68-933961a6e2ec None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.716456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.717183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.717476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.717758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.718542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:33.718861 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:33.720325 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:33.730609 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 181/356] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:33.731565 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.743247 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e56459-87df-4301-87d3-5d91887c5842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.744152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e56459-87df-4301-87d3-5d91887c5842 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.744445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e56459-87df-4301-87d3-5d91887c5842 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.744731 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e56459-87df-4301-87d3-5d91887c5842 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.746049 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e56459-87df-4301-87d3-5d91887c5842 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:33.747576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e56459-87df-4301-87d3-5d91887c5842 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:33.749750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 176/357] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:33.751163 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeafb8-9645-40ef-8b6e-e417a99e261e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.751463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeafb8-9645-40ef-8b6e-e417a99e261e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.751746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeafb8-9645-40ef-8b6e-e417a99e261e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.753313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 177/358] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:33.754034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedeafb8-9645-40ef-8b6e-e417a99e261e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.837345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab7ca8-a60c-420c-8774-a425e691c563 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.865365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedeafb8-9645-40ef-8b6e-e417a99e261e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.872064 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f9471-4281-4e6e-b7b0-96e79796501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.872956 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f9471-4281-4e6e-b7b0-96e79796501a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.873303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f9471-4281-4e6e-b7b0-96e79796501a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.873559 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f9471-4281-4e6e-b7b0-96e79796501a None admin] PATH_INFO: `/v3/users/4fb1abe2c3324055977c2e70f9e3bbd3` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.874796 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f9471-4281-4e6e-b7b0-96e79796501a None admin] RBAC: Authorizing `identity:get_user(user_id=4fb1abe2c3324055977c2e70f9e3bbd3)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:33.876240 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f9471-4281-4e6e-b7b0-96e79796501a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:33.878237 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 182/359] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/users/4fb1abe2c3324055977c2e70f9e3bbd3 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:33.879024 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f9471-4281-4e6e-b7b0-96e79796501a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:33.906144 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66b8a-5161-478c-840f-ee7e6fa4a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:33.906144 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66b8a-5161-478c-840f-ee7e6fa4a819 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:33.906144 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66b8a-5161-478c-840f-ee7e6fa4a819 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:33.906815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66b8a-5161-478c-840f-ee7e6fa4a819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:33.989866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f9471-4281-4e6e-b7b0-96e79796501a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:33.993853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66b8a-5161-478c-840f-ee7e6fa4a819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:33.995345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66b8a-5161-478c-840f-ee7e6fa4a819 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:34.028917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619b896-9163-4fc6-8d66-5705e39f6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:34.029910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b896-9163-4fc6-8d66-5705e39f6468 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.030203 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b896-9163-4fc6-8d66-5705e39f6468 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.030651 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619b896-9163-4fc6-8d66-5705e39f6468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.032119 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619b896-9163-4fc6-8d66-5705e39f6468 None admin] RBAC: Adding query filter params (group.id=8537bbbb4fcf49d0a56e6eaa390573ce, role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=1322e64cad064e79a31ef6484516a1de) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:34.032460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619b896-9163-4fc6-8d66-5705e39f6468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:34.034165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619b896-9163-4fc6-8d66-5705e39f6468 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:34.042545 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 183/360] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/role_assignments?group.id=8537bbbb4fcf49d0a56e6eaa390573ce&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:34.044579 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06300b1e-9d39-4063-ab9e-abd4bf5ea51d None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.046277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06300b1e-9d39-4063-ab9e-abd4bf5ea51d None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.046620 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06300b1e-9d39-4063-ab9e-abd4bf5ea51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.071222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-06300b1e-9d39-4063-ab9e-abd4bf5ea51d None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:34.072517 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-06300b1e-9d39-4063-ab9e-abd4bf5ea51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:34.125767 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 178/361] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:34.127512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66b8a-5161-478c-840f-ee7e6fa4a819 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:34.164771 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 184/362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:34.165714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06300b1e-9d39-4063-ab9e-abd4bf5ea51d None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:34.272578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66b8a-5161-478c-840f-ee7e6fa4a819 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:34.340361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:34.341068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.341348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.341630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.342478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] RBAC: Adding query filter params (role.id=9178c520b46b41e48fc1659fe0129123, user.id=4fb1abe2c3324055977c2e70f9e3bbd3, scope.system=all) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:34.342796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:34.344248 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:34.353704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06300b1e-9d39-4063-ab9e-abd4bf5ea51d None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:34.359731 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 179/363] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 9 15:41:34 2022] GET /identity/v3/role_assignments?user.id=4fb1abe2c3324055977c2e70f9e3bbd3&scope.system=all&role.id=9178c520b46b41e48fc1659fe0129123 => generated 510 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:34.360595 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:34.397534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02381e-2b2e-47b0-b956-1d9aa1975e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:34.398884 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02381e-2b2e-47b0-b956-1d9aa1975e6b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.399493 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02381e-2b2e-47b0-b956-1d9aa1975e6b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.400024 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02381e-2b2e-47b0-b956-1d9aa1975e6b None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.401741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02381e-2b2e-47b0-b956-1d9aa1975e6b None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:34.404067 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02381e-2b2e-47b0-b956-1d9aa1975e6b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:34.407096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 185/364] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:34 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:34.409101 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701c830-584a-4d23-8876-a8bb329d8da4 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.409774 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701c830-584a-4d23-8876-a8bb329d8da4 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.410396 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701c830-584a-4d23-8876-a8bb329d8da4 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.412552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 186/365] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:34.417409 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701c830-584a-4d23-8876-a8bb329d8da4 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:34.532043 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed4ea5-75d6-4415-99d9-96a49c2a8b9c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:34.573803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c6e45-966f-453a-9446-182820d9bfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:34.574658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c6e45-966f-453a-9446-182820d9bfb1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.574957 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c6e45-966f-453a-9446-182820d9bfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.575251 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c6e45-966f-453a-9446-182820d9bfb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.599956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170c6e45-966f-453a-9446-182820d9bfb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-170c6e45-966f-453a-9446-182820d9bfb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:34.608603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:34.609736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:34.610674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:34.619777 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 180/366] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:34.622691 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c169-c3ce-4b73-b66e-8f120b6076fb None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.623902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c169-c3ce-4b73-b66e-8f120b6076fb None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.625032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c169-c3ce-4b73-b66e-8f120b6076fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.631091 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701c830-584a-4d23-8876-a8bb329d8da4 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:34.653866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-5807c169-c3ce-4b73-b66e-8f120b6076fb None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:34.655011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-5807c169-c3ce-4b73-b66e-8f120b6076fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:34.684685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493213-91a0-497f-9424-1ed1a0eadc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:34.686427 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493213-91a0-497f-9424-1ed1a0eadc7a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.687067 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493213-91a0-497f-9424-1ed1a0eadc7a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.687668 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493213-91a0-497f-9424-1ed1a0eadc7a None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.690383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493213-91a0-497f-9424-1ed1a0eadc7a None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:34.693181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493213-91a0-497f-9424-1ed1a0eadc7a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:34.695764 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 187/367] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:34 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:34.696902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493213-91a0-497f-9424-1ed1a0eadc7a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:34.744544 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 181/368] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:34.745255 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807c169-c3ce-4b73-b66e-8f120b6076fb None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:34.831703 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493213-91a0-497f-9424-1ed1a0eadc7a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:34.860356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807c169-c3ce-4b73-b66e-8f120b6076fb None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:34.875954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:34.877391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.878159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.878668 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.880129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:34.880662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:34.882262 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:34.894221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 188/369] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:34.895795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:34.896757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:34.897517 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:34.897812 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:34.898157 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:34.899152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] RBAC: Adding query filter params (group.id=8537bbbb4fcf49d0a56e6eaa390573ce, role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:34.899476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:34.901655 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:34.913273 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 182/370] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:34 2022] GET /identity/v3/role_assignments?group.id=8537bbbb4fcf49d0a56e6eaa390573ce&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:34.916902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:35.032194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a40f7-5e19-454d-8bab-a8c694695075 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:35.050816 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea8f6b-62c0-4ff5-8119-e28dccc846cb None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:35.077215 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55e2d2-109d-4325-a44c-6af494326544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:35.078486 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e2d2-109d-4325-a44c-6af494326544 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:35.078988 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e2d2-109d-4325-a44c-6af494326544 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:35.079488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55e2d2-109d-4325-a44c-6af494326544 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:35.088043 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966f2a6-fa43-4725-b0f6-b7ed114deb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:35.088915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966f2a6-fa43-4725-b0f6-b7ed114deb98 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:35.089127 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966f2a6-fa43-4725-b0f6-b7ed114deb98 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:35.089411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966f2a6-fa43-4725-b0f6-b7ed114deb98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:35.094104 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55e2d2-109d-4325-a44c-6af494326544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-1b55e2d2-109d-4325-a44c-6af494326544 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:35.103333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:35.104437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:35.105400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:35.108290 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 189/371] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:35.114144 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55e2d2-109d-4325-a44c-6af494326544 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:35.172785 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966f2a6-fa43-4725-b0f6-b7ed114deb98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:35.174306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966f2a6-fa43-4725-b0f6-b7ed114deb98 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:35.284590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55e2d2-109d-4325-a44c-6af494326544 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:35.341561 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:35.342255 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:35.342544 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:35.342827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:35.343594 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:35.343910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:35.345345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:35.365183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 190/373] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:35.365389 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 183/373] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:35.371461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:35.371629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966f2a6-fa43-4725-b0f6-b7ed114deb98 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:35.480851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966f2a6-fa43-4725-b0f6-b7ed114deb98 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:35.523691 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a415-3574-4422-a374-08a7e2bb248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:35.524556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a415-3574-4422-a374-08a7e2bb248f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:35.524847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a415-3574-4422-a374-08a7e2bb248f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:35.525157 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a415-3574-4422-a374-08a7e2bb248f None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:35.529778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a415-3574-4422-a374-08a7e2bb248f None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:35.531243 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a415-3574-4422-a374-08a7e2bb248f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:35.533603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 184/374] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:35 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:35.539202 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a415-3574-4422-a374-08a7e2bb248f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:35.560276 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e46ca2-c6ca-49ec-9c67-5cbd6ad2a36d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:35.603582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c04bb6-db9f-4e99-8e6b-6564f148c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:35.604614 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c04bb6-db9f-4e99-8e6b-6564f148c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:35.605138 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c04bb6-db9f-4e99-8e6b-6564f148c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:35.605642 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c04bb6-db9f-4e99-8e6b-6564f148c3a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:35.641797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a415-3574-4422-a374-08a7e2bb248f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:35.689816 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4902d-aa62-46d5-8898-2e758bd26b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:35.690702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4902d-aa62-46d5-8898-2e758bd26b32 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:35.691064 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4902d-aa62-46d5-8898-2e758bd26b32 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:35.691262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4902d-aa62-46d5-8898-2e758bd26b32 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:35.692917 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4902d-aa62-46d5-8898-2e758bd26b32 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:35.694047 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4902d-aa62-46d5-8898-2e758bd26b32 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:35.695383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c04bb6-db9f-4e99-8e6b-6564f148c3a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:35.696008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 185/375] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:35 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:35.697383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c04bb6-db9f-4e99-8e6b-6564f148c3a6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:35.701827 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4902d-aa62-46d5-8898-2e758bd26b32 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:35.868607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 191/376] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:35.873960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c04bb6-db9f-4e99-8e6b-6564f148c3a6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:35.903880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4902d-aa62-46d5-8898-2e758bd26b32 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:35.983791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c04bb6-db9f-4e99-8e6b-6564f148c3a6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:35.990365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:35.991052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:35.991342 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:35.991626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:35.992487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] RBAC: Adding query filter params (group.id=8537bbbb4fcf49d0a56e6eaa390573ce, role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=1322e64cad064e79a31ef6484516a1de) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:35.992809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:35.994316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:36.004055 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 186/377] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:35 2022] GET /identity/v3/role_assignments?group.id=8537bbbb4fcf49d0a56e6eaa390573ce&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=044a2d93445743af91a96573d39f21eb => generated 260 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:36.028416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c973db-73fc-4fd7-9d4f-cfea60515fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:36.029773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c973db-73fc-4fd7-9d4f-cfea60515fc2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:36.030292 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c973db-73fc-4fd7-9d4f-cfea60515fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:36.030814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c973db-73fc-4fd7-9d4f-cfea60515fc2 None admin] PATH_INFO: `/v3/projects/3c3add198476495285e8aab3267a1d15` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:36.038061 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c973db-73fc-4fd7-9d4f-cfea60515fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=3c3add198476495285e8aab3267a1d15)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:36.040419 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c973db-73fc-4fd7-9d4f-cfea60515fc2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:36.042898 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 192/378] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:35 2022] GET /identity/v3/projects/3c3add198476495285e8aab3267a1d15 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:36.048417 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:36.199326 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65607337-68f9-417b-8aba-24bf9b60d047 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:36.238800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822f5f4-e1d9-4fda-b490-5e5d68b194d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:36.239708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f5f4-e1d9-4fda-b490-5e5d68b194d6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:36.239996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f5f4-e1d9-4fda-b490-5e5d68b194d6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:36.240277 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f5f4-e1d9-4fda-b490-5e5d68b194d6 None admin] PATH_INFO: `/v3/groups/675aefde6bd7469baecf86958600e55e` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:36.246342 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822f5f4-e1d9-4fda-b490-5e5d68b194d6 None admin] RBAC: Authorizing `identity:get_group(group_id=675aefde6bd7469baecf86958600e55e)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:36.247808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822f5f4-e1d9-4fda-b490-5e5d68b194d6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:36.249846 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 187/379] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:36 2022] GET /identity/v3/groups/675aefde6bd7469baecf86958600e55e => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:36.255528 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c973db-73fc-4fd7-9d4f-cfea60515fc2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:36.373807 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93295-17f8-4463-8a3d-2a357273fce0 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:36.374139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93295-17f8-4463-8a3d-2a357273fce0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:36.374392 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb93295-17f8-4463-8a3d-2a357273fce0 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:36.376034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 188/380] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:36.381627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d317509-3138-447b-82d5-4346d3ea90f0 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:36.381924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d317509-3138-447b-82d5-4346d3ea90f0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:36.382215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d317509-3138-447b-82d5-4346d3ea90f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:36.396180 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c973db-73fc-4fd7-9d4f-cfea60515fc2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:36.401574 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-0d317509-3138-447b-82d5-4346d3ea90f0 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:36.402776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-0d317509-3138-447b-82d5-4346d3ea90f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:36.455783 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:36.456374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:36.456668 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:36.456957 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:36.458467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] RBAC: Adding query filter params (group.id=675aefde6bd7469baecf86958600e55e, role.id=ebff922955b74c919d58613875326151, scope.project.id=3c3add198476495285e8aab3267a1d15) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:36.458747 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:36.460199 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:36.478035 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 189/381] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:36.479556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 193/382] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:36 2022] GET /identity/v3/role_assignments?group.id=675aefde6bd7469baecf86958600e55e&scope.project.id=3c3add198476495285e8aab3267a1d15&role.id=ebff922955b74c919d58613875326151 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:36.501607 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d317509-3138-447b-82d5-4346d3ea90f0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:36.615391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d317509-3138-447b-82d5-4346d3ea90f0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:36.653455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ddc25-30bb-4f41-b47d-1e5e31f3b27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:36.654285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ddc25-30bb-4f41-b47d-1e5e31f3b27f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:36.654581 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ddc25-30bb-4f41-b47d-1e5e31f3b27f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:36.654864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ddc25-30bb-4f41-b47d-1e5e31f3b27f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:36.666214 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ddc25-30bb-4f41-b47d-1e5e31f3b27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5b0ddc25-30bb-4f41-b47d-1e5e31f3b27f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:36.674631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:36.675743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:36.676709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 190/383] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:36.682250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:36.790303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7cf506-072d-438a-ba34-73436d470112 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:36.827166 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:36.827996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:36.828256 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:36.828495 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:36.829306 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:36.829621 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:36.831113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:36.840739 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 194/384] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:36.846490 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ddc25-30bb-4f41-b47d-1e5e31f3b27f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:36.951210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ddc25-30bb-4f41-b47d-1e5e31f3b27f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:36.985550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c752-374f-45e0-b1d6-e5cd70f92ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:36.986425 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c752-374f-45e0-b1d6-e5cd70f92ead None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:36.986714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c752-374f-45e0-b1d6-e5cd70f92ead None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:36.986996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c752-374f-45e0-b1d6-e5cd70f92ead None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:36.988213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797c752-374f-45e0-b1d6-e5cd70f92ead None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:36.990297 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797c752-374f-45e0-b1d6-e5cd70f92ead None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:36.992363 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 191/385] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:36 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:36.997794 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:37.107555 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b32896-f3c0-400c-88f4-dbea22fa64a6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:37.143352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee7e52-5b9d-4608-b197-f4455b3b3482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:37.144217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee7e52-5b9d-4608-b197-f4455b3b3482 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:37.144508 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee7e52-5b9d-4608-b197-f4455b3b3482 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:37.144793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee7e52-5b9d-4608-b197-f4455b3b3482 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:37.146075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee7e52-5b9d-4608-b197-f4455b3b3482 None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:37.147542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee7e52-5b9d-4608-b197-f4455b3b3482 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:37.149574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 195/386] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:36 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:37.155252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c752-374f-45e0-b1d6-e5cd70f92ead None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:37.238539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1e23d-654f-4f22-986a-96a82da4f27e None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:37.238876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1e23d-654f-4f22-986a-96a82da4f27e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:37.240016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1e23d-654f-4f22-986a-96a82da4f27e None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:37.242003 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 196/387] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:37.249420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5924b-ea16-4402-a9c3-5bddad3f02e9 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:37.249723 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5924b-ea16-4402-a9c3-5bddad3f02e9 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:37.250329 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5924b-ea16-4402-a9c3-5bddad3f02e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:37.271984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-ecd5924b-ea16-4402-a9c3-5bddad3f02e9 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:37.272585 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c752-374f-45e0-b1d6-e5cd70f92ead None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:37.273256 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-ecd5924b-ea16-4402-a9c3-5bddad3f02e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:37.322345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f952bf-c9f4-448d-9cdd-1e35892e3be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:37.323245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f952bf-c9f4-448d-9cdd-1e35892e3be7 None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:37.323534 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f952bf-c9f4-448d-9cdd-1e35892e3be7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:37.323815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f952bf-c9f4-448d-9cdd-1e35892e3be7 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/groups/8537bbbb4fcf49d0a56e6eaa390573ce/roles/044a2d93445743af91a96573d39f21eb` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:37.329854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f952bf-c9f4-448d-9cdd-1e35892e3be7 None admin] RBAC: Authorizing `identity:create_grant(project_id=38e69abc282f4bf1aa231ecc15fc213b, group_id=8537bbbb4fcf49d0a56e6eaa390573ce, role_id=044a2d93445743af91a96573d39f21eb)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:37.331333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f952bf-c9f4-448d-9cdd-1e35892e3be7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:37.356518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 192/388] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:41:37 2022] PUT /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/groups/8537bbbb4fcf49d0a56e6eaa390573ce/roles/044a2d93445743af91a96573d39f21eb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:37.368696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 197/389] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:37.392252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f952bf-c9f4-448d-9cdd-1e35892e3be7 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:37.507860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f952bf-c9f4-448d-9cdd-1e35892e3be7 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:37.547338 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23880e3-13ef-4f19-909e-867282546e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:37.548161 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23880e3-13ef-4f19-909e-867282546e21 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:37.548451 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23880e3-13ef-4f19-909e-867282546e21 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:37.548732 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23880e3-13ef-4f19-909e-867282546e21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:37.561604 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23880e3-13ef-4f19-909e-867282546e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-d23880e3-13ef-4f19-909e-867282546e21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:37.569755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:37.570800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:37.571733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 193/390] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:37.577241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5924b-ea16-4402-a9c3-5bddad3f02e9 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:37.700282 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5924b-ea16-4402-a9c3-5bddad3f02e9 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:37.746830 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:37.747912 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:37.748426 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:37.748919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:37.749917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:37.750431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:37.752034 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:37.764007 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 198/391] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:37.770046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23880e3-13ef-4f19-909e-867282546e21 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:37.892682 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23880e3-13ef-4f19-909e-867282546e21 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:37.930138 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39135149-5be4-40a8-b5d2-ed449317e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:37.931038 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39135149-5be4-40a8-b5d2-ed449317e519 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:37.931330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39135149-5be4-40a8-b5d2-ed449317e519 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:37.931615 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39135149-5be4-40a8-b5d2-ed449317e519 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:37.932946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39135149-5be4-40a8-b5d2-ed449317e519 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:37.934593 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39135149-5be4-40a8-b5d2-ed449317e519 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:37.936709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 194/392] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:37 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:37.943077 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:38.071836 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad543c7-f7d2-490d-8e16-978c3ec14e34 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:38.117822 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c127d-3b73-4bbf-b278-8fa78630b584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:38.118725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c127d-3b73-4bbf-b278-8fa78630b584 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:38.119019 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c127d-3b73-4bbf-b278-8fa78630b584 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:38.119303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c127d-3b73-4bbf-b278-8fa78630b584 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:38.120614 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c127d-3b73-4bbf-b278-8fa78630b584 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:38.122097 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c127d-3b73-4bbf-b278-8fa78630b584 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:38.123989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 199/393] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:37 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:38.130964 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39135149-5be4-40a8-b5d2-ed449317e519 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:38.255832 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39135149-5be4-40a8-b5d2-ed449317e519 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:38.295199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21422a7-d50a-4900-ba7e-2d005907186b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:38.296099 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21422a7-d50a-4900-ba7e-2d005907186b None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:38.296386 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21422a7-d50a-4900-ba7e-2d005907186b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:38.296669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21422a7-d50a-4900-ba7e-2d005907186b None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de/groups/8537bbbb4fcf49d0a56e6eaa390573ce/roles/22b9b0cd5e41452380f6aff61caa9f91` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:38.298759 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21422a7-d50a-4900-ba7e-2d005907186b None admin] RBAC: Authorizing `identity:create_grant(project_id=1322e64cad064e79a31ef6484516a1de, group_id=8537bbbb4fcf49d0a56e6eaa390573ce, role_id=22b9b0cd5e41452380f6aff61caa9f91)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:38.300220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21422a7-d50a-4900-ba7e-2d005907186b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:38.314585 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca723f-23b9-4c66-ae32-3f3209102e0b None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:38.314875 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca723f-23b9-4c66-ae32-3f3209102e0b None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:38.315168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca723f-23b9-4c66-ae32-3f3209102e0b None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:38.316725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 200/394] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:38.326110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 195/395] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:41:38 2022] PUT /identity/v3/projects/1322e64cad064e79a31ef6484516a1de/groups/8537bbbb4fcf49d0a56e6eaa390573ce/roles/22b9b0cd5e41452380f6aff61caa9f91 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:41:38.331044 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369be80-efed-4b35-9903-367bcdcf4005 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:38.331340 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369be80-efed-4b35-9903-367bcdcf4005 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:38.331625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369be80-efed-4b35-9903-367bcdcf4005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:38.359033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-2369be80-efed-4b35-9903-367bcdcf4005 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:38.360274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-2369be80-efed-4b35-9903-367bcdcf4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:38.438800 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 201/396] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:38.475708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21422a7-d50a-4900-ba7e-2d005907186b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:38.623634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21422a7-d50a-4900-ba7e-2d005907186b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:38.678739 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629a17d-d2c6-4dde-a2bb-11e547e05f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:38.679959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629a17d-d2c6-4dde-a2bb-11e547e05f41 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:38.680490 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629a17d-d2c6-4dde-a2bb-11e547e05f41 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:38.681154 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629a17d-d2c6-4dde-a2bb-11e547e05f41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:38.697964 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629a17d-d2c6-4dde-a2bb-11e547e05f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-3629a17d-d2c6-4dde-a2bb-11e547e05f41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:38.706367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:38.707391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:38.708397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:38.715813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 196/397] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:38.716579 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369be80-efed-4b35-9903-367bcdcf4005 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:38.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369be80-efed-4b35-9903-367bcdcf4005 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:38.884288 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:38.884855 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:38.885198 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:38.885481 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:38.886264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:38.886590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:38.888013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:38.898148 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 202/398] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:38.904135 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629a17d-d2c6-4dde-a2bb-11e547e05f41 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:39.032060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629a17d-d2c6-4dde-a2bb-11e547e05f41 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:39.072678 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de52f0-b2ae-40fb-8c37-948c4cfc14e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:39.073590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de52f0-b2ae-40fb-8c37-948c4cfc14e9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.073884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de52f0-b2ae-40fb-8c37-948c4cfc14e9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.074172 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de52f0-b2ae-40fb-8c37-948c4cfc14e9 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.075412 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de52f0-b2ae-40fb-8c37-948c4cfc14e9 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:39.076876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de52f0-b2ae-40fb-8c37-948c4cfc14e9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:39.078914 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 197/399] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:38 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:39.084606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:39.186255 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c21bf-d327-4295-9c0a-8edcfc9c858c None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.186617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c21bf-d327-4295-9c0a-8edcfc9c858c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.186831 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c21bf-d327-4295-9c0a-8edcfc9c858c None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.188264 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 198/400] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:41:39.202592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4004c-cad1-4869-af5a-f87a7e3a1d41 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.203284 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4004c-cad1-4869-af5a-f87a7e3a1d41 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.203803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4004c-cad1-4869-af5a-f87a7e3a1d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.223989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee8efb-9549-4578-89d0-10d6c9548042 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:39.240542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-50d4004c-cad1-4869-af5a-f87a7e3a1d41 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:39.242153 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-50d4004c-cad1-4869-af5a-f87a7e3a1d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:39.263917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a588db6-b024-444b-a0bc-2ba02a1cd2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:39.265055 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a588db6-b024-444b-a0bc-2ba02a1cd2dc None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.265383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a588db6-b024-444b-a0bc-2ba02a1cd2dc None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.265666 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a588db6-b024-444b-a0bc-2ba02a1cd2dc None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.268285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a588db6-b024-444b-a0bc-2ba02a1cd2dc None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:39.271462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a588db6-b024-444b-a0bc-2ba02a1cd2dc None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:39.274316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 203/401] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:39 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:39.282200 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a588db6-b024-444b-a0bc-2ba02a1cd2dc None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:39.391782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 199/402] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:39.419831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a588db6-b024-444b-a0bc-2ba02a1cd2dc None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:39.437663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4004c-cad1-4869-af5a-f87a7e3a1d41 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:39.467231 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc815fbc-e6a8-4895-b065-30aadc423f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:39.468140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc815fbc-e6a8-4895-b065-30aadc423f19 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.468428 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc815fbc-e6a8-4895-b065-30aadc423f19 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.468715 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc815fbc-e6a8-4895-b065-30aadc423f19 None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/groups/8537bbbb4fcf49d0a56e6eaa390573ce/roles/22b9b0cd5e41452380f6aff61caa9f91` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.473424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc815fbc-e6a8-4895-b065-30aadc423f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=38e69abc282f4bf1aa231ecc15fc213b, group_id=8537bbbb4fcf49d0a56e6eaa390573ce, role_id=22b9b0cd5e41452380f6aff61caa9f91)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:39.474901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc815fbc-e6a8-4895-b065-30aadc423f19 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:39.496626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 204/403] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:41:39 2022] PUT /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b/groups/8537bbbb4fcf49d0a56e6eaa390573ce/roles/22b9b0cd5e41452380f6aff61caa9f91 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:41:39.575600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff96b-3ef8-4495-a76b-09a2f2258be7 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.576194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff96b-3ef8-4495-a76b-09a2f2258be7 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.576644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff96b-3ef8-4495-a76b-09a2f2258be7 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.581720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4004c-cad1-4869-af5a-f87a7e3a1d41 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:39.581952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 205/404] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:41:39.585713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc395a-3975-4385-b3ae-f1ecea3e3173 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.586338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc395a-3975-4385-b3ae-f1ecea3e3173 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.586466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc395a-3975-4385-b3ae-f1ecea3e3173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.622861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931267-c75b-427e-8746-e0b18a4f29d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:39.623739 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931267-c75b-427e-8746-e0b18a4f29d8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.624092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931267-c75b-427e-8746-e0b18a4f29d8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.624470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931267-c75b-427e-8746-e0b18a4f29d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.630287 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-b5dc395a-3975-4385-b3ae-f1ecea3e3173 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:39.631569 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-b5dc395a-3975-4385-b3ae-f1ecea3e3173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:39.640080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b931267-c75b-427e-8746-e0b18a4f29d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-6b931267-c75b-427e-8746-e0b18a4f29d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:39.652942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:39.654345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:39.655513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:39.658783 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 200/405] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:39.665244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931267-c75b-427e-8746-e0b18a4f29d8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:39.755056 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 206/406] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:39.798820 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc395a-3975-4385-b3ae-f1ecea3e3173 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:39.827223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931267-c75b-427e-8746-e0b18a4f29d8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:39.877974 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:39.879238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:39.879722 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:39.880178 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:39.881352 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:39.881985 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:39.883568 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:39.894922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 201/407] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:39.901326 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:40.017213 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc395a-3975-4385-b3ae-f1ecea3e3173 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:40.026186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877b6da-f9f3-4c29-a100-ea6e14e705a1 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:40.080032 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50199f-0939-474f-b22f-467e8c086dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:40.081700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50199f-0939-474f-b22f-467e8c086dc8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.082403 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50199f-0939-474f-b22f-467e8c086dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.083893 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50199f-0939-474f-b22f-467e8c086dc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.092539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2316bb-27a8-4090-b979-98c76d401457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:40.093694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2316bb-27a8-4090-b979-98c76d401457 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.094207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2316bb-27a8-4090-b979-98c76d401457 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.094658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2316bb-27a8-4090-b979-98c76d401457 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.096196 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2316bb-27a8-4090-b979-98c76d401457 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:40.098291 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2316bb-27a8-4090-b979-98c76d401457 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:40.101150 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 202/408] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:39 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:40.109157 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2316bb-27a8-4090-b979-98c76d401457 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:40.114003 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50199f-0939-474f-b22f-467e8c086dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-bf50199f-0939-474f-b22f-467e8c086dc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:40.125072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:40.126318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:40.127369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:40.128078 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 207/409] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:40.129119 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50199f-0939-474f-b22f-467e8c086dc8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:40.298877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50199f-0939-474f-b22f-467e8c086dc8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:40.335507 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2316bb-27a8-4090-b979-98c76d401457 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:40.343428 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675428-e315-48d1-aec5-f25083449120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:40.344407 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675428-e315-48d1-aec5-f25083449120 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.344876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675428-e315-48d1-aec5-f25083449120 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.345358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07675428-e315-48d1-aec5-f25083449120 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.346306 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07675428-e315-48d1-aec5-f25083449120 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:40.346779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07675428-e315-48d1-aec5-f25083449120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:40.348647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07675428-e315-48d1-aec5-f25083449120 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:40.358901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 208/410] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:40.364785 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675428-e315-48d1-aec5-f25083449120 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:40.394297 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99251c1-d90e-4b74-a5d5-c234290071a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:40.395208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99251c1-d90e-4b74-a5d5-c234290071a9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.396115 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99251c1-d90e-4b74-a5d5-c234290071a9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.396368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99251c1-d90e-4b74-a5d5-c234290071a9 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.397665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99251c1-d90e-4b74-a5d5-c234290071a9 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:40.399124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99251c1-d90e-4b74-a5d5-c234290071a9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:40.406388 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 203/411] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:40 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:40.411774 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99251c1-d90e-4b74-a5d5-c234290071a9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:40.471284 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07675428-e315-48d1-aec5-f25083449120 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:40.512444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1449277-f640-4942-b728-43806ab70925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:40.513614 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1449277-f640-4942-b728-43806ab70925 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.513914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1449277-f640-4942-b728-43806ab70925 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.514200 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1449277-f640-4942-b728-43806ab70925 None admin] PATH_INFO: `/v3/groups/675aefde6bd7469baecf86958600e55e` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.515452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1449277-f640-4942-b728-43806ab70925 None admin] RBAC: Authorizing `identity:get_group(group_id=675aefde6bd7469baecf86958600e55e)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:40.517004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1449277-f640-4942-b728-43806ab70925 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:40.518983 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 209/412] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:40 2022] GET /identity/v3/groups/675aefde6bd7469baecf86958600e55e => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:40.524478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff07ce-a6f6-4477-b4b7-347f3caa0db4 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.524478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff07ce-a6f6-4477-b4b7-347f3caa0db4 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.524478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff07ce-a6f6-4477-b4b7-347f3caa0db4 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.524478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 210/413] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:40.525449 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff07ce-a6f6-4477-b4b7-347f3caa0db4 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:40.569630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99251c1-d90e-4b74-a5d5-c234290071a9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:40.606862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4a005-b277-48c8-9703-11fde46afd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:40.608157 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a005-b277-48c8-9703-11fde46afd6f None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.608705 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a005-b277-48c8-9703-11fde46afd6f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.609235 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a005-b277-48c8-9703-11fde46afd6f None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de/groups/8537bbbb4fcf49d0a56e6eaa390573ce/roles/044a2d93445743af91a96573d39f21eb` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.611811 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4a005-b277-48c8-9703-11fde46afd6f None admin] RBAC: Authorizing `identity:create_grant(project_id=1322e64cad064e79a31ef6484516a1de, group_id=8537bbbb4fcf49d0a56e6eaa390573ce, role_id=044a2d93445743af91a96573d39f21eb)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:40.613549 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4a005-b277-48c8-9703-11fde46afd6f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:40.634252 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff07ce-a6f6-4477-b4b7-347f3caa0db4 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:40.638789 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 204/414] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:41:40 2022] PUT /identity/v3/projects/1322e64cad064e79a31ef6484516a1de/groups/8537bbbb4fcf49d0a56e6eaa390573ce/roles/044a2d93445743af91a96573d39f21eb => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:41:40.644417 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171760d-d3fd-4f72-9f38-3dff2113c311 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.644749 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171760d-d3fd-4f72-9f38-3dff2113c311 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.644998 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171760d-d3fd-4f72-9f38-3dff2113c311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.672007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-f171760d-d3fd-4f72-9f38-3dff2113c311 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:40.673337 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-f171760d-d3fd-4f72-9f38-3dff2113c311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:40.741812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd43a19-ac49-440b-b97e-f9bfb3bcedea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:40.742719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43a19-ac49-440b-b97e-f9bfb3bcedea None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.743016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43a19-ac49-440b-b97e-f9bfb3bcedea None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.743308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd43a19-ac49-440b-b97e-f9bfb3bcedea None admin] PATH_INFO: `/v3/projects/3c3add198476495285e8aab3267a1d15` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.745123 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd43a19-ac49-440b-b97e-f9bfb3bcedea None admin] RBAC: Authorizing `identity:get_project(project_id=3c3add198476495285e8aab3267a1d15)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:40.746205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd43a19-ac49-440b-b97e-f9bfb3bcedea None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:40.752589 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 211/415] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:40 2022] GET /identity/v3/projects/3c3add198476495285e8aab3267a1d15 => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:40.753937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd43a19-ac49-440b-b97e-f9bfb3bcedea None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:40.765710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 205/416] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:40.804811 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171760d-d3fd-4f72-9f38-3dff2113c311 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:40.886596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd43a19-ac49-440b-b97e-f9bfb3bcedea None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:40.925687 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e61e67-f3cb-449f-a245-1776a04a232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:40.927176 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e61e67-f3cb-449f-a245-1776a04a232f None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:40.927473 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e61e67-f3cb-449f-a245-1776a04a232f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:40.927766 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e61e67-f3cb-449f-a245-1776a04a232f None admin] PATH_INFO: `/v3/projects/3c3add198476495285e8aab3267a1d15/groups/675aefde6bd7469baecf86958600e55e/roles/ebff922955b74c919d58613875326151` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:40.931180 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e61e67-f3cb-449f-a245-1776a04a232f None admin] RBAC: Authorizing `identity:create_grant(project_id=3c3add198476495285e8aab3267a1d15, group_id=675aefde6bd7469baecf86958600e55e, role_id=ebff922955b74c919d58613875326151)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:40.932652 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e61e67-f3cb-449f-a245-1776a04a232f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:40.953561 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 212/417] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 9 15:41:40 2022] PUT /identity/v3/projects/3c3add198476495285e8aab3267a1d15/groups/675aefde6bd7469baecf86958600e55e/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:41:40.964378 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171760d-d3fd-4f72-9f38-3dff2113c311 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:41.008905 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd01f3-b8c9-41f5-bb3b-0b20262e8cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:41.009806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd01f3-b8c9-41f5-bb3b-0b20262e8cac None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:41.010096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd01f3-b8c9-41f5-bb3b-0b20262e8cac None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:41.010374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd01f3-b8c9-41f5-bb3b-0b20262e8cac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:41.026854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd01f3-b8c9-41f5-bb3b-0b20262e8cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-11fd01f3-b8c9-41f5-bb3b-0b20262e8cac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:41.035332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:41.037000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:41.038506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 206/418] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:41.043193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e61e67-f3cb-449f-a245-1776a04a232f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:41.188014 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e61e67-f3cb-449f-a245-1776a04a232f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:41.224124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:41.224871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:41.225208 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:41.225493 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:41.226268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:41.226592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:41.227980 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:41.237152 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 213/419] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:41.244023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd01f3-b8c9-41f5-bb3b-0b20262e8cac None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:41.352259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd01f3-b8c9-41f5-bb3b-0b20262e8cac None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:41.394201 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e63f3-c3ce-4513-bd81-d0e2432133f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:41.394867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e63f3-c3ce-4513-bd81-d0e2432133f1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:41.395507 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e63f3-c3ce-4513-bd81-d0e2432133f1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:41.395507 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e63f3-c3ce-4513-bd81-d0e2432133f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:41.471344 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e63f3-c3ce-4513-bd81-d0e2432133f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:41.473029 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e63f3-c3ce-4513-bd81-d0e2432133f1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:41.601385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 207/420] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:41:41.606572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:41.745874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e0fbf-d048-4098-88c3-ce3ab83ae1cf None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:41.788925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69560-b06e-4855-b803-6e758d63036f None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:41.789276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69560-b06e-4855-b803-6e758d63036f None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:41.789564 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf69560-b06e-4855-b803-6e758d63036f None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:41.792418 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 208/421] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:41.799499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c663949-ec5a-4a9c-9a1b-2a69d727821d None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:41.799821 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c663949-ec5a-4a9c-9a1b-2a69d727821d None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:41.800117 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c663949-ec5a-4a9c-9a1b-2a69d727821d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:41.803636 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c30326-60d9-4ab5-8612-f5fe110ee86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:41.804517 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c30326-60d9-4ab5-8612-f5fe110ee86b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:41.804807 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c30326-60d9-4ab5-8612-f5fe110ee86b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:41.805127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c30326-60d9-4ab5-8612-f5fe110ee86b None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:41.807416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c30326-60d9-4ab5-8612-f5fe110ee86b None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:41.808865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c30326-60d9-4ab5-8612-f5fe110ee86b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:41.814105 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 214/422] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:41 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:41.819351 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c30326-60d9-4ab5-8612-f5fe110ee86b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:41.828056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-7c663949-ec5a-4a9c-9a1b-2a69d727821d None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:41.829325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-7c663949-ec5a-4a9c-9a1b-2a69d727821d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:41.959351 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 209/423] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:41.985416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c30326-60d9-4ab5-8612-f5fe110ee86b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:42.001400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c663949-ec5a-4a9c-9a1b-2a69d727821d None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:42.048628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ffabb-70c9-45ac-8d92-7435c614b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:42.049814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ffabb-70c9-45ac-8d92-7435c614b631 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.052195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ffabb-70c9-45ac-8d92-7435c614b631 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.052663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ffabb-70c9-45ac-8d92-7435c614b631 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.054085 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ffabb-70c9-45ac-8d92-7435c614b631 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:42.055690 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ffabb-70c9-45ac-8d92-7435c614b631 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:42.057909 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 215/424] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:41 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:42.063217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ffabb-70c9-45ac-8d92-7435c614b631 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:42.185571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c663949-ec5a-4a9c-9a1b-2a69d727821d None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:42.212390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ffabb-70c9-45ac-8d92-7435c614b631 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:42.250519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:42.251527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.251981 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.252452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.253520 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] RBAC: Adding query filter params (group.id=8537bbbb4fcf49d0a56e6eaa390573ce, role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:42.254001 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:42.255673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:42.267956 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 216/425] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:42 2022] GET /identity/v3/role_assignments?group.id=8537bbbb4fcf49d0a56e6eaa390573ce&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=044a2d93445743af91a96573d39f21eb => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:42.279392 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384934b-f5fd-4c6a-aef1-bc87a785cc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:42.280234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384934b-f5fd-4c6a-aef1-bc87a785cc27 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.280537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384934b-f5fd-4c6a-aef1-bc87a785cc27 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.280822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384934b-f5fd-4c6a-aef1-bc87a785cc27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.303760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384934b-f5fd-4c6a-aef1-bc87a785cc27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-f384934b-f5fd-4c6a-aef1-bc87a785cc27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:42.311776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:42.312891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:42.314910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 210/426] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:42.319150 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:42.418650 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897725ef-48d4-45ec-8c42-41c5a9f316f4 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.419233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897725ef-48d4-45ec-8c42-41c5a9f316f4 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.419233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897725ef-48d4-45ec-8c42-41c5a9f316f4 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.420859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 211/427] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:42.431096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065d392-101e-497b-b786-1cd594bf8bfe None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.431096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065d392-101e-497b-b786-1cd594bf8bfe None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.431430 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065d392-101e-497b-b786-1cd594bf8bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.465926 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582be110-faf9-4ccf-9621-0df4195c2a17 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:42.466723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-d065d392-101e-497b-b786-1cd594bf8bfe None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:42.467962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-d065d392-101e-497b-b786-1cd594bf8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:42.505718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:42.506830 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.507353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.507859 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.509001 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] RBAC: Adding query filter params (name=member) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:42.509654 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:42.511309 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:42.522199 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 217/428] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:42.527827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:42.590103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 212/429] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:42.616027 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065d392-101e-497b-b786-1cd594bf8bfe None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:42.662367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c89ce1-b9c2-4afd-bfc3-19ec12024848 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:42.698608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1a41-5782-4fb8-ba82-4048c109b2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:42.699647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1a41-5782-4fb8-ba82-4048c109b2cc None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.700195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1a41-5782-4fb8-ba82-4048c109b2cc None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.700710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1a41-5782-4fb8-ba82-4048c109b2cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.725376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065d392-101e-497b-b786-1cd594bf8bfe None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:42.769065 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bdd63-2695-4690-b10b-c04bbabbf8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:42.769820 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bdd63-2695-4690-b10b-c04bbabbf8a4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.770643 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bdd63-2695-4690-b10b-c04bbabbf8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.770978 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bdd63-2695-4690-b10b-c04bbabbf8a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.785277 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bdd63-2695-4690-b10b-c04bbabbf8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:42.786912 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f1a41-5782-4fb8-ba82-4048c109b2cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:42.788489 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f1a41-5782-4fb8-ba82-4048c109b2cc None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-c55bdd63-2695-4690-b10b-c04bbabbf8a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:42.794697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:42.795741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:42.796720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 213/430] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:42.802508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bdd63-2695-4690-b10b-c04bbabbf8a4 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:42.895875 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 218/431] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:42.905058 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1a41-5782-4fb8-ba82-4048c109b2cc None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:42.925371 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bdd63-2695-4690-b10b-c04bbabbf8a4 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:42.961703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:42.962804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:42.963333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:42.964057 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:42.965027 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] RBAC: Adding query filter params (name=member) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:42.965514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:42.967653 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:42.984279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 214/432] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:41:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:42.985962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.026339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1a41-5782-4fb8-ba82-4048c109b2cc None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.069071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4db5d-5372-4cb2-aa5c-ef280b05d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.070937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4db5d-5372-4cb2-aa5c-ef280b05d584 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.071346 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4db5d-5372-4cb2-aa5c-ef280b05d584 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.071746 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4db5d-5372-4cb2-aa5c-ef280b05d584 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.073140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4db5d-5372-4cb2-aa5c-ef280b05d584 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:43.074701 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4db5d-5372-4cb2-aa5c-ef280b05d584 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:43.081080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 219/433] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:42 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:43.083652 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4db5d-5372-4cb2-aa5c-ef280b05d584 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.110046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6390fb-b827-48f1-883d-5a952d9f9ce0 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.144963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520655df-3d92-4315-a0ba-03c9c2f74c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.146210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520655df-3d92-4315-a0ba-03c9c2f74c19 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.146557 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520655df-3d92-4315-a0ba-03c9c2f74c19 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.146840 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520655df-3d92-4315-a0ba-03c9c2f74c19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.218586 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520655df-3d92-4315-a0ba-03c9c2f74c19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:43.220060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4db5d-5372-4cb2-aa5c-ef280b05d584 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.220189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520655df-3d92-4315-a0ba-03c9c2f74c19 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:43.264790 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4aaf39-c7cc-49e5-bcda-e5f3c07ce3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.265558 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4aaf39-c7cc-49e5-bcda-e5f3c07ce3a8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.265847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4aaf39-c7cc-49e5-bcda-e5f3c07ce3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.266129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4aaf39-c7cc-49e5-bcda-e5f3c07ce3a8 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.267336 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4aaf39-c7cc-49e5-bcda-e5f3c07ce3a8 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:43.268791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4aaf39-c7cc-49e5-bcda-e5f3c07ce3a8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:43.276050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 220/434] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:43 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:43.277055 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4aaf39-c7cc-49e5-bcda-e5f3c07ce3a8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.348269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 215/435] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:43.354697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520655df-3d92-4315-a0ba-03c9c2f74c19 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.378137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4aaf39-c7cc-49e5-bcda-e5f3c07ce3a8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.414396 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b80d8-31b0-4808-ba5c-dfbc549e1ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.415130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b80d8-31b0-4808-ba5c-dfbc549e1ac8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.415421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b80d8-31b0-4808-ba5c-dfbc549e1ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.415725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b80d8-31b0-4808-ba5c-dfbc549e1ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.416574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b80d8-31b0-4808-ba5c-dfbc549e1ac8 None admin] RBAC: Adding query filter params (group.id=8537bbbb4fcf49d0a56e6eaa390573ce, role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=1322e64cad064e79a31ef6484516a1de) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:43.416890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b80d8-31b0-4808-ba5c-dfbc549e1ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:43.418365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b80d8-31b0-4808-ba5c-dfbc549e1ac8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:43.429074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 221/436] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:43 2022] GET /identity/v3/role_assignments?group.id=8537bbbb4fcf49d0a56e6eaa390573ce&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:43.429074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c515c-a4b1-4e38-9f42-02784007ae92 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.429074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c515c-a4b1-4e38-9f42-02784007ae92 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.429074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c515c-a4b1-4e38-9f42-02784007ae92 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.429074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 222/437] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:43.432900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5c0b7-c9d2-47c2-9e26-dfc285143831 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.433229 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5c0b7-c9d2-47c2-9e26-dfc285143831 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.433519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5c0b7-c9d2-47c2-9e26-dfc285143831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.452033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-6cf5c0b7-c9d2-47c2-9e26-dfc285143831 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:43.453302 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-6cf5c0b7-c9d2-47c2-9e26-dfc285143831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:43.490556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520655df-3d92-4315-a0ba-03c9c2f74c19 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.536769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b79f-5e9a-4069-8e7a-85a46fa0478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.537628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b79f-5e9a-4069-8e7a-85a46fa0478e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.537919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b79f-5e9a-4069-8e7a-85a46fa0478e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.538202 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92b79f-5e9a-4069-8e7a-85a46fa0478e None admin] PATH_INFO: `/v3/projects/38e69abc282f4bf1aa231ecc15fc213b` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.539547 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92b79f-5e9a-4069-8e7a-85a46fa0478e None admin] RBAC: Authorizing `identity:get_project(project_id=38e69abc282f4bf1aa231ecc15fc213b)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:43.541295 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92b79f-5e9a-4069-8e7a-85a46fa0478e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:43.547537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 216/438] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:43 2022] GET /identity/v3/projects/38e69abc282f4bf1aa231ecc15fc213b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:43.550443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b79f-5e9a-4069-8e7a-85a46fa0478e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.561137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 223/439] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:43.587484 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5c0b7-c9d2-47c2-9e26-dfc285143831 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.671450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92b79f-5e9a-4069-8e7a-85a46fa0478e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.703125 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5c0b7-c9d2-47c2-9e26-dfc285143831 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.708666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d1649-f03e-4ae2-83a2-9048c345af45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.709590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d1649-f03e-4ae2-83a2-9048c345af45 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.709886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d1649-f03e-4ae2-83a2-9048c345af45 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.710170 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d1649-f03e-4ae2-83a2-9048c345af45 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.711401 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d1649-f03e-4ae2-83a2-9048c345af45 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:43.712868 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d1649-f03e-4ae2-83a2-9048c345af45 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:43.719349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 217/440] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:43 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:43.720616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39884-3285-4934-abb3-3d0d72b3a978 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.720945 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39884-3285-4934-abb3-3d0d72b3a978 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.721317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39884-3285-4934-abb3-3d0d72b3a978 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.722808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 218/441] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:43.723900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e39884-3285-4934-abb3-3d0d72b3a978 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.739599 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375ba29-9fa4-42fe-90ad-86afd299311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.740411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375ba29-9fa4-42fe-90ad-86afd299311d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.740692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375ba29-9fa4-42fe-90ad-86afd299311d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.740964 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375ba29-9fa4-42fe-90ad-86afd299311d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.750772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375ba29-9fa4-42fe-90ad-86afd299311d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-1375ba29-9fa4-42fe-90ad-86afd299311d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:43.759421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:43.760463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:43.761467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 224/442] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:43.762253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59010460-9950-464c-b4a5-89ec26c16c44 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.762253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59010460-9950-464c-b4a5-89ec26c16c44 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.762406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59010460-9950-464c-b4a5-89ec26c16c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.778136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-59010460-9950-464c-b4a5-89ec26c16c44 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:43.779307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-59010460-9950-464c-b4a5-89ec26c16c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:43.823516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e39884-3285-4934-abb3-3d0d72b3a978 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.843886 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 225/443] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:43.844781 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59010460-9950-464c-b4a5-89ec26c16c44 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.861592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.862266 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.862560 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.862852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.863710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] RBAC: Adding query filter params (group.id=8537bbbb4fcf49d0a56e6eaa390573ce, role.id=22b9b0cd5e41452380f6aff61caa9f91, scope.project.id=38e69abc282f4bf1aa231ecc15fc213b) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:43.864545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:43.866063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:43.872141 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 219/444] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:43 2022] GET /identity/v3/role_assignments?group.id=8537bbbb4fcf49d0a56e6eaa390573ce&scope.project.id=38e69abc282f4bf1aa231ecc15fc213b&role.id=22b9b0cd5e41452380f6aff61caa9f91 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:43.873023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:43.957491 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59010460-9950-464c-b4a5-89ec26c16c44 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.990930 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73430c48-43b6-4bda-a53b-ea6924ec8d40 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:43.994094 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:43.994788 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:43.995081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:43.995354 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:43.996127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:43.996441 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:43.997878 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.011855 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 226/445] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:44.012645 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.028597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d089df-d65e-471e-a4c2-afc85ed8245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.029464 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d089df-d65e-471e-a4c2-afc85ed8245a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.029752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d089df-d65e-471e-a4c2-afc85ed8245a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.030033 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d089df-d65e-471e-a4c2-afc85ed8245a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.041886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d089df-d65e-471e-a4c2-afc85ed8245a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-50d089df-d65e-471e-a4c2-afc85ed8245a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:44.049714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:44.050842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:44.052644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 220/446] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:44.057837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d089df-d65e-471e-a4c2-afc85ed8245a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.128158 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424fa5a-4e75-43e8-9d2d-5e432a39e6a6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.164067 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc02f-31bd-4c7b-9715-35724106330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.164721 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc02f-31bd-4c7b-9715-35724106330c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.165008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc02f-31bd-4c7b-9715-35724106330c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.165242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d089df-d65e-471e-a4c2-afc85ed8245a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.165350 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc02f-31bd-4c7b-9715-35724106330c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.202560 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.203012 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.203297 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.203574 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.204327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:44.204637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.206117 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.214831 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 221/447] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:44.220449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.237196 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fc02f-31bd-4c7b-9715-35724106330c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.238682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fc02f-31bd-4c7b-9715-35724106330c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.330887 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa3b49-4a7e-4ba2-b823-49961db39c45 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.349086 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 227/448] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:41:44.354174 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc02f-31bd-4c7b-9715-35724106330c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.369419 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4a0f-7ed6-47ad-b5c3-c97fb850d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.370095 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4a0f-7ed6-47ad-b5c3-c97fb850d93b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.371138 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4a0f-7ed6-47ad-b5c3-c97fb850d93b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.371429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec4a0f-7ed6-47ad-b5c3-c97fb850d93b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.451008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4a0f-7ed6-47ad-b5c3-c97fb850d93b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.452514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec4a0f-7ed6-47ad-b5c3-c97fb850d93b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.459527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc02f-31bd-4c7b-9715-35724106330c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.494114 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd190db5-24dc-42b1-b630-1cd306c8f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.494983 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190db5-24dc-42b1-b630-1cd306c8f295 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.495262 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190db5-24dc-42b1-b630-1cd306c8f295 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.495538 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190db5-24dc-42b1-b630-1cd306c8f295 None admin] PATH_INFO: `/v3/projects/1322e64cad064e79a31ef6484516a1de` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.496812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd190db5-24dc-42b1-b630-1cd306c8f295 None admin] RBAC: Authorizing `identity:get_project(project_id=1322e64cad064e79a31ef6484516a1de)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.500021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd190db5-24dc-42b1-b630-1cd306c8f295 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.500021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 228/449] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/projects/1322e64cad064e79a31ef6484516a1de => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:44.506145 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd190db5-24dc-42b1-b630-1cd306c8f295 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.564344 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 222/450] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:41:44.569787 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4a0f-7ed6-47ad-b5c3-c97fb850d93b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.606155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd190db5-24dc-42b1-b630-1cd306c8f295 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.638713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f519a4e-da4f-41af-b15d-4d79412bb1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.639559 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519a4e-da4f-41af-b15d-4d79412bb1f3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.639843 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519a4e-da4f-41af-b15d-4d79412bb1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.640121 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519a4e-da4f-41af-b15d-4d79412bb1f3 None admin] PATH_INFO: `/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.641344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519a4e-da4f-41af-b15d-4d79412bb1f3 None admin] RBAC: Authorizing `identity:get_group(group_id=8537bbbb4fcf49d0a56e6eaa390573ce)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.642794 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519a4e-da4f-41af-b15d-4d79412bb1f3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.644409 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 229/451] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/groups/8537bbbb4fcf49d0a56e6eaa390573ce => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:44.650081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f519a4e-da4f-41af-b15d-4d79412bb1f3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.664697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec4a0f-7ed6-47ad-b5c3-c97fb850d93b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.697607 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaee92-64e1-4bf9-9bdf-9300dd6f204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.698470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaee92-64e1-4bf9-9bdf-9300dd6f204b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.698790 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaee92-64e1-4bf9-9bdf-9300dd6f204b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.699133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaee92-64e1-4bf9-9bdf-9300dd6f204b None admin] PATH_INFO: `/v3/projects/3c3add198476495285e8aab3267a1d15` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.700461 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaee92-64e1-4bf9-9bdf-9300dd6f204b None admin] RBAC: Authorizing `identity:get_project(project_id=3c3add198476495285e8aab3267a1d15)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.701984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaee92-64e1-4bf9-9bdf-9300dd6f204b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.703632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 223/452] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/projects/3c3add198476495285e8aab3267a1d15 => generated 344 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:44.709556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaee92-64e1-4bf9-9bdf-9300dd6f204b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.743705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f519a4e-da4f-41af-b15d-4d79412bb1f3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.776174 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.776825 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.777133 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.777414 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.778248 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] RBAC: Adding query filter params (group.id=8537bbbb4fcf49d0a56e6eaa390573ce, role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=1322e64cad064e79a31ef6484516a1de) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:44.778564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.779985 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.785899 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 230/453] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/role_assignments?group.id=8537bbbb4fcf49d0a56e6eaa390573ce&scope.project.id=1322e64cad064e79a31ef6484516a1de&role.id=044a2d93445743af91a96573d39f21eb => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:44.803663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaee92-64e1-4bf9-9bdf-9300dd6f204b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.840483 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196ac83-66eb-46c2-b9d2-ecfde41170d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.841436 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ac83-66eb-46c2-b9d2-ecfde41170d3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.841729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ac83-66eb-46c2-b9d2-ecfde41170d3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.842016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ac83-66eb-46c2-b9d2-ecfde41170d3 None admin] PATH_INFO: `/v3/groups/675aefde6bd7469baecf86958600e55e` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.843333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196ac83-66eb-46c2-b9d2-ecfde41170d3 None admin] RBAC: Authorizing `identity:get_group(group_id=675aefde6bd7469baecf86958600e55e)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.844848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196ac83-66eb-46c2-b9d2-ecfde41170d3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:44.846630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 224/454] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/groups/675aefde6bd7469baecf86958600e55e => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:44.853085 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:44.958875 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7b5d0-8168-4168-984d-2c78cb981ebb None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:44.993271 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604ad81-f904-40fc-8d4b-63d8eef74ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:44.993928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604ad81-f904-40fc-8d4b-63d8eef74ab0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:44.994216 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604ad81-f904-40fc-8d4b-63d8eef74ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:44.994495 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604ad81-f904-40fc-8d4b-63d8eef74ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:44.995334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604ad81-f904-40fc-8d4b-63d8eef74ab0 None admin] RBAC: Adding query filter params (group.id=675aefde6bd7469baecf86958600e55e, role.id=ebff922955b74c919d58613875326151, scope.project.id=3c3add198476495285e8aab3267a1d15) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:44.995646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604ad81-f904-40fc-8d4b-63d8eef74ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:44.997097 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604ad81-f904-40fc-8d4b-63d8eef74ab0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:45.002819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 231/455] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 9 15:41:44 2022] GET /identity/v3/role_assignments?group.id=675aefde6bd7469baecf86958600e55e&scope.project.id=3c3add198476495285e8aab3267a1d15&role.id=ebff922955b74c919d58613875326151 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:47.634753 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebc257-9839-4ba4-a015-2b9b74d4d7db None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:47.635821 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebc257-9839-4ba4-a015-2b9b74d4d7db None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:47.636356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebc257-9839-4ba4-a015-2b9b74d4d7db None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:47.638148 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 225/456] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:47.644236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333c2e1-e418-451e-b968-bad23858010a None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:47.645722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333c2e1-e418-451e-b968-bad23858010a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:47.646227 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333c2e1-e418-451e-b968-bad23858010a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:47.672276 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-1333c2e1-e418-451e-b968-bad23858010a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:47.673882 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-1333c2e1-e418-451e-b968-bad23858010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:47.775630 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 232/457] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:47.813224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebc257-9839-4ba4-a015-2b9b74d4d7db None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.005923 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebc257-9839-4ba4-a015-2b9b74d4d7db None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:48.027879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f12818-4eb1-48bd-a1f5-7927795f7903 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.028578 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f12818-4eb1-48bd-a1f5-7927795f7903 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.028927 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f12818-4eb1-48bd-a1f5-7927795f7903 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.030805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 233/458] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:48.047856 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cbc9b-f262-4f0a-84b7-56b2e4115238 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.048142 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cbc9b-f262-4f0a-84b7-56b2e4115238 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.048638 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cbc9b-f262-4f0a-84b7-56b2e4115238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.056487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:48.057344 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.057634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.057921 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.080931 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:48.082468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:48.085222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-5b9cbc9b-f262-4f0a-84b7-56b2e4115238 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:48.086444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-5b9cbc9b-f262-4f0a-84b7-56b2e4115238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:48.109542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:48.110591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:48.111577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:48.111577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:48.111577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:48.111577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:48.111577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 226/459] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:48.118630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.158527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 234/460] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:48.166378 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ec62-8387-4f16-aaf5-7a3eab063dda None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.166685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ec62-8387-4f16-aaf5-7a3eab063dda None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.166978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9ec62-8387-4f16-aaf5-7a3eab063dda None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.168585 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 235/461] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:41:48.182454 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739101-8b1c-48af-848f-68fef6ec3da8 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.182713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739101-8b1c-48af-848f-68fef6ec3da8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.182946 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739101-8b1c-48af-848f-68fef6ec3da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.200450 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-f7739101-8b1c-48af-848f-68fef6ec3da8 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:48.201745 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-f7739101-8b1c-48af-848f-68fef6ec3da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:48.253481 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbbaaf-bd11-489a-9c9c-441d52f20b34 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:48.273391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 236/462] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:48.274271 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7739101-8b1c-48af-848f-68fef6ec3da8 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.314853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:48.315538 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.315826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.316107 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.316869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:48.317811 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:48.319211 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:48.337426 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 227/463] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:48.338300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.386337 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7739101-8b1c-48af-848f-68fef6ec3da8 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:48.423015 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:48.423826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.424171 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.424465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.435687 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:48.437135 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:48.441080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11474a96-4429-4ecc-a4c5-55c7f1d67788 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:48.455296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:48.457602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:48.459891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:48.459891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:48.459891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:48.459891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:48.459891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 237/464] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:48.459891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.475957 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:48.476765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.477065 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.477376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.489424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:48.490485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:48.510167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:48.512704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:48.514907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:48.514907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:48.514907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:48.514907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:48.514907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 228/465] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:48.514907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.568099 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91eda0e-302d-4c15-8a67-ad5833c06acd None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:48.601823 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6f628-da03-4a2f-9e71-b193eff888d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:48.602503 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6f628-da03-4a2f-9e71-b193eff888d9 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.602793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6f628-da03-4a2f-9e71-b193eff888d9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.603076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6f628-da03-4a2f-9e71-b193eff888d9 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.603793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6f628-da03-4a2f-9e71-b193eff888d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:48.605284 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6f628-da03-4a2f-9e71-b193eff888d9 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:48.615611 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18ee6a-9836-4dfd-89b5-c4131d586e69 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:48.624860 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 238/466] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:48 2022] POST /identity/v3/users => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:48.625952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6f628-da03-4a2f-9e71-b193eff888d9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.667946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c071f8a-b82c-4ab1-9a51-3d9a5c1e29e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:48.668684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c071f8a-b82c-4ab1-9a51-3d9a5c1e29e9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.668941 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c071f8a-b82c-4ab1-9a51-3d9a5c1e29e9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.669261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c071f8a-b82c-4ab1-9a51-3d9a5c1e29e9 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.670693 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c071f8a-b82c-4ab1-9a51-3d9a5c1e29e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:48.671033 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c071f8a-b82c-4ab1-9a51-3d9a5c1e29e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:48.672501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c071f8a-b82c-4ab1-9a51-3d9a5c1e29e9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:48.696775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 229/467] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:48.696775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4520c8-8c58-48be-a97d-2cfae881da94 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.696775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4520c8-8c58-48be-a97d-2cfae881da94 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.696775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4520c8-8c58-48be-a97d-2cfae881da94 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.696775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 230/468] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:48.698949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4520c8-8c58-48be-a97d-2cfae881da94 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.763050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6f628-da03-4a2f-9e71-b193eff888d9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:48.812209 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f53d84-25ae-4168-b9a4-d47db673471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:48.813411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f53d84-25ae-4168-b9a4-d47db673471f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.813933 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f53d84-25ae-4168-b9a4-d47db673471f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.814306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4520c8-8c58-48be-a97d-2cfae881da94 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:48.814678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f53d84-25ae-4168-b9a4-d47db673471f None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.816272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f53d84-25ae-4168-b9a4-d47db673471f None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:48.816943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f53d84-25ae-4168-b9a4-d47db673471f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:48.818599 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f53d84-25ae-4168-b9a4-d47db673471f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:48.831355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 239/469] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:48.833591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71be75-a707-448a-bf52-e6a9175d23b8 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.834068 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71be75-a707-448a-bf52-e6a9175d23b8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.834517 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71be75-a707-448a-bf52-e6a9175d23b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.855381 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-5b71be75-a707-448a-bf52-e6a9175d23b8 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:48.856330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6ea1-4518-4b75-9f8d-17b090ac0a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:48.857773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-5b71be75-a707-448a-bf52-e6a9175d23b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:48.858603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6ea1-4518-4b75-9f8d-17b090ac0a8c None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:48.859054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6ea1-4518-4b75-9f8d-17b090ac0a8c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:48.861597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6ea1-4518-4b75-9f8d-17b090ac0a8c None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:48.863183 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac6ea1-4518-4b75-9f8d-17b090ac0a8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:48.864682 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac6ea1-4518-4b75-9f8d-17b090ac0a8c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:48.889470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 231/470] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:48 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:48.890454 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6ea1-4518-4b75-9f8d-17b090ac0a8c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:48.955485 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 240/471] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:48.997644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71be75-a707-448a-bf52-e6a9175d23b8 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:49.021067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6ea1-4518-4b75-9f8d-17b090ac0a8c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:49.057132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9c0f0-29fd-44b4-a308-e99eeee90fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:49.057132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9c0f0-29fd-44b4-a308-e99eeee90fe5 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:49.057132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9c0f0-29fd-44b4-a308-e99eeee90fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:49.057132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9c0f0-29fd-44b4-a308-e99eeee90fe5 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:49.057678 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9c0f0-29fd-44b4-a308-e99eeee90fe5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:49.057678 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9c0f0-29fd-44b4-a308-e99eeee90fe5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:49.090903 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 232/472] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:48 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:49.136141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71be75-a707-448a-bf52-e6a9175d23b8 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:49.180751 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:49.181909 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:49.182454 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:49.182914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:49.195059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:49.196888 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:49.220408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:49.221529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:49.222842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:49.222842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:49.222842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:49.222842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:49.222842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 241/473] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:49.228790 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9c0f0-29fd-44b4-a308-e99eeee90fe5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:49.373220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9c0f0-29fd-44b4-a308-e99eeee90fe5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:49.417132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f86826-3b7d-4886-ad7c-3b8f8a52fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:49.417132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f86826-3b7d-4886-ad7c-3b8f8a52fd5e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:49.417132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f86826-3b7d-4886-ad7c-3b8f8a52fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:49.417132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f86826-3b7d-4886-ad7c-3b8f8a52fd5e None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:49.417578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f86826-3b7d-4886-ad7c-3b8f8a52fd5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:49.417578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f86826-3b7d-4886-ad7c-3b8f8a52fd5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:49.417578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f86826-3b7d-4886-ad7c-3b8f8a52fd5e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:49.435908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 233/474] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:49.441066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:49.478958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754805eb-006f-4ace-ad6a-3b3585b1a489 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:49.479262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754805eb-006f-4ace-ad6a-3b3585b1a489 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:49.479455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754805eb-006f-4ace-ad6a-3b3585b1a489 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:49.481120 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 234/475] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:49.487205 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb5013-88ce-4acb-894b-34bdcb52938e None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:49.487496 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb5013-88ce-4acb-894b-34bdcb52938e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:49.487773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb5013-88ce-4acb-894b-34bdcb52938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:49.507326 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-c5fb5013-88ce-4acb-894b-34bdcb52938e None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:49.508817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-c5fb5013-88ce-4acb-894b-34bdcb52938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:49.553732 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd9ab7-830f-41cc-86fa-9946352586e7 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:49.594736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961ee5d-bbf0-4959-bf2a-bf8e74f1db0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:49.595418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961ee5d-bbf0-4959-bf2a-bf8e74f1db0f None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:49.595703 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961ee5d-bbf0-4959-bf2a-bf8e74f1db0f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:49.595985 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961ee5d-bbf0-4959-bf2a-bf8e74f1db0f None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:49.596835 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961ee5d-bbf0-4959-bf2a-bf8e74f1db0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:49.599201 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961ee5d-bbf0-4959-bf2a-bf8e74f1db0f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:49.601133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 235/476] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:49.623741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb5013-88ce-4acb-894b-34bdcb52938e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:49.643902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 242/477] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:49 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:49.799129 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb5013-88ce-4acb-894b-34bdcb52938e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:49.847135 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:49.848659 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:49.848952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:49.849267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:49.850063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:49.851493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:49.865461 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.sql.core [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:41:49.865461 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 15:41:49.865461 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [parameters: {'id': 'b4e8b693811a4c458f2a234a740fede8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 15:41:49.865461 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=275711) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application conn.query(q) May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result.read() May 09 15:41:49.882124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return ref.to_dict() May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:41:49.883197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application next(self.gen) May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application yield resource May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application next(self.gen) May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise self.value May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application session.commit() May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:41:49.884760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.session.flush() May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application compat.raise_( May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:41:49.885809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:41:49.886880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application conn.query(q) May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result.read() May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:41:49.887845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [parameters: {'id': 'b4e8b693811a4c458f2a234a740fede8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:49.888794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 09 15:41:49.891573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:49.893804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 236/478] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:41:49 2022] POST /identity/v3/roles => generated 144 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:49.906371 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961ee5d-bbf0-4959-bf2a-bf8e74f1db0f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:50.025352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961ee5d-bbf0-4959-bf2a-bf8e74f1db0f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:50.060793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b325a-35a8-4adf-9506-cb1c888b2d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:50.061612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b325a-35a8-4adf-9506-cb1c888b2d52 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:50.061904 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b325a-35a8-4adf-9506-cb1c888b2d52 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:50.062192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b325a-35a8-4adf-9506-cb1c888b2d52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:50.073446 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b325a-35a8-4adf-9506-cb1c888b2d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-f39b325a-35a8-4adf-9506-cb1c888b2d52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:50.081773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:50.083463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 09 15:41:50.084393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:50.088293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 243/479] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:41:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:50.090155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:50.205772 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114986d3-4d2f-455e-bca8-15f7d26ce008 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:50.248442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc483dd8-d50f-46ca-abbe-848b5748344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:50.249148 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483dd8-d50f-46ca-abbe-848b5748344b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:50.249438 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483dd8-d50f-46ca-abbe-848b5748344b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:50.249722 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483dd8-d50f-46ca-abbe-848b5748344b None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:50.250477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc483dd8-d50f-46ca-abbe-848b5748344b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:50.250790 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc483dd8-d50f-46ca-abbe-848b5748344b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:50.252158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc483dd8-d50f-46ca-abbe-848b5748344b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:50.261088 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 237/480] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:51.227845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d58bb-df1a-4883-86b4-cbbdf0bdb731 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:51.228347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d58bb-df1a-4883-86b4-cbbdf0bdb731 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:51.228347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d58bb-df1a-4883-86b4-cbbdf0bdb731 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:51.230026 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 244/481] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:51.235017 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd36bf-4f83-48d7-a1c3-3ed0f32cfdb6 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:51.235407 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd36bf-4f83-48d7-a1c3-3ed0f32cfdb6 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:51.235593 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd36bf-4f83-48d7-a1c3-3ed0f32cfdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:51.257393 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-10bd36bf-4f83-48d7-a1c3-3ed0f32cfdb6 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:51.258596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-10bd36bf-4f83-48d7-a1c3-3ed0f32cfdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:51.344043 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 238/482] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:51.375712 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d58bb-df1a-4883-86b4-cbbdf0bdb731 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:51.528781 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d58bb-df1a-4883-86b4-cbbdf0bdb731 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:51.577652 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacd18f-5913-4afc-9576-1acd2c579022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:51.578867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacd18f-5913-4afc-9576-1acd2c579022 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:51.579376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacd18f-5913-4afc-9576-1acd2c579022 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:51.579881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacd18f-5913-4afc-9576-1acd2c579022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:51.598172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacd18f-5913-4afc-9576-1acd2c579022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-8eacd18f-5913-4afc-9576-1acd2c579022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:51.606490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:51.607526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:51.608462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:51.611020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 245/483] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:51.618166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd36bf-4f83-48d7-a1c3-3ed0f32cfdb6 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:51.761259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd36bf-4f83-48d7-a1c3-3ed0f32cfdb6 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:51.795026 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:51.795633 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:51.795925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:51.796208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:51.796973 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:51.797313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:51.798886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:51.800648 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20de1d-91eb-4c07-9a7b-7c30a76f357f None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:51.801756 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20de1d-91eb-4c07-9a7b-7c30a76f357f None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:51.802272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20de1d-91eb-4c07-9a7b-7c30a76f357f None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:51.804128 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 246/484] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:51.808650 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 239/485] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:41:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:51.810022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baac0e5-7e95-44c9-8877-cbbb89d331ea None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:51.810530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baac0e5-7e95-44c9-8877-cbbb89d331ea None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:51.811055 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baac0e5-7e95-44c9-8877-cbbb89d331ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:51.817411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:51.850029 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-7baac0e5-7e95-44c9-8877-cbbb89d331ea None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:51.851663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-7baac0e5-7e95-44c9-8877-cbbb89d331ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:51.934728 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161cacc-af17-4f8f-9715-600ec083f848 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:51.982599 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:51.983434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:51.983730 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:51.984019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.006579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:52.008068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:52.015114 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 247/486] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:52.031708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:52.032733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:52.033765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:52.033765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:52.033765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:52.033765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:52.037311 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 240/487] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:52.038829 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baac0e5-7e95-44c9-8877-cbbb89d331ea None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.058631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.174871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baac0e5-7e95-44c9-8877-cbbb89d331ea None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.206225 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750740-1866-4b7e-a87c-bbd3dcd2030a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.217171 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f761b2f-fc08-42cf-bcb2-1e895f4f437e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.217859 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f761b2f-fc08-42cf-bcb2-1e895f4f437e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.218150 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f761b2f-fc08-42cf-bcb2-1e895f4f437e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.218436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f761b2f-fc08-42cf-bcb2-1e895f4f437e None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.219558 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f761b2f-fc08-42cf-bcb2-1e895f4f437e None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:52.219882 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f761b2f-fc08-42cf-bcb2-1e895f4f437e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:52.221693 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f761b2f-fc08-42cf-bcb2-1e895f4f437e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:52.238135 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 248/488] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:52.240101 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61097ff7-2953-444a-961e-01677c134a9a None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.240338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61097ff7-2953-444a-961e-01677c134a9a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.240628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61097ff7-2953-444a-961e-01677c134a9a None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.246510 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 249/489] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:52.247268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61097ff7-2953-444a-961e-01677c134a9a None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.252446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3965d6b-c718-42c0-a82d-b26b473d8930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.253882 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965d6b-c718-42c0-a82d-b26b473d8930 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.254948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965d6b-c718-42c0-a82d-b26b473d8930 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.255443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965d6b-c718-42c0-a82d-b26b473d8930 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.269413 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3965d6b-c718-42c0-a82d-b26b473d8930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-f3965d6b-c718-42c0-a82d-b26b473d8930 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:52.282596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:52.283866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:52.284984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:52.287921 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 241/490] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:52.289994 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9721-4d19-4db2-bde1-7fce5e7d66ff None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.290294 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9721-4d19-4db2-bde1-7fce5e7d66ff None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.290588 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce9721-4d19-4db2-bde1-7fce5e7d66ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.311025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-8dce9721-4d19-4db2-bde1-7fce5e7d66ff None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:52.312723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-8dce9721-4d19-4db2-bde1-7fce5e7d66ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:52.372204 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61097ff7-2953-444a-961e-01677c134a9a None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.387590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 242/491] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:52.388527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce9721-4d19-4db2-bde1-7fce5e7d66ff None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.415678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.416683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.417129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.417519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] PATH_INFO: `/v3/users/nova` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.425775 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:52.427420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:41:52.445465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:52.446819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:41:52.448246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:52.450006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 250/492] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:52.450910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.517705 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce9721-4d19-4db2-bde1-7fce5e7d66ff None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.553319 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.554238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.554728 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.555196 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.556140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:52.556668 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:52.558285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:52.562880 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda317e-cddf-4a84-9572-68e4a99c1556 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.568050 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 243/493] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:52.569302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.599340 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71931c75-12ed-47c8-aea2-10e3a9a0b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.600283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71931c75-12ed-47c8-aea2-10e3a9a0b206 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.600646 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71931c75-12ed-47c8-aea2-10e3a9a0b206 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.601004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71931c75-12ed-47c8-aea2-10e3a9a0b206 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.611820 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71931c75-12ed-47c8-aea2-10e3a9a0b206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-71931c75-12ed-47c8-aea2-10e3a9a0b206 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:52.620612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:52.621834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:52.623192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 251/494] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:52.624020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71931c75-12ed-47c8-aea2-10e3a9a0b206 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.675143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977b41a-a7d5-4a29-b400-9ffb20df4b86 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.708532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b6e5c-c4c0-4d44-bf2b-53cc08f07790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.709250 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6e5c-c4c0-4d44-bf2b-53cc08f07790 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.709547 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6e5c-c4c0-4d44-bf2b-53cc08f07790 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.709835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b6e5c-c4c0-4d44-bf2b-53cc08f07790 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.710701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b6e5c-c4c0-4d44-bf2b-53cc08f07790 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:52.711025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b6e5c-c4c0-4d44-bf2b-53cc08f07790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:52.712464 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b6e5c-c4c0-4d44-bf2b-53cc08f07790 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:52.723130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71931c75-12ed-47c8-aea2-10e3a9a0b206 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.749014 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 244/495] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:52.750384 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c89a0-7f76-4b59-95be-9a0b1ad73905 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.750681 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c89a0-7f76-4b59-95be-9a0b1ad73905 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.750965 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c89a0-7f76-4b59-95be-9a0b1ad73905 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.756906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 245/496] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:52.756906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c89a0-7f76-4b59-95be-9a0b1ad73905 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.765024 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.765848 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.766136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.766428 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.777144 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:52.778617 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:52.795850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:52.796914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:52.797978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:52.797978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:52.797978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:52.797978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:52.797978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 252/497] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:52.798297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:52.865475 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c89a0-7f76-4b59-95be-9a0b1ad73905 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.901758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4ebb0-0fe0-4474-a784-41cfe6160008 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:52.906042 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781c6af-012d-4a81-91f5-8c63494208d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.906734 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781c6af-012d-4a81-91f5-8c63494208d1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.907022 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781c6af-012d-4a81-91f5-8c63494208d1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.907304 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781c6af-012d-4a81-91f5-8c63494208d1 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.908072 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781c6af-012d-4a81-91f5-8c63494208d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:52.908385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781c6af-012d-4a81-91f5-8c63494208d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:52.910260 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781c6af-012d-4a81-91f5-8c63494208d1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:52.920186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 246/498] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:52.921738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02223b-23e5-41b0-a8d2-1ec87d83ed60 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.922182 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02223b-23e5-41b0-a8d2-1ec87d83ed60 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.922632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02223b-23e5-41b0-a8d2-1ec87d83ed60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:52.939452 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-dd02223b-23e5-41b0-a8d2-1ec87d83ed60 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:52.940869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-dd02223b-23e5-41b0-a8d2-1ec87d83ed60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:52.943618 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00888b40-8f57-48dd-893b-8106cd0d65f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:52.944277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888b40-8f57-48dd-893b-8106cd0d65f0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:52.944569 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888b40-8f57-48dd-893b-8106cd0d65f0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:52.944854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888b40-8f57-48dd-893b-8106cd0d65f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.008669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 247/499] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:53.009401 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02223b-23e5-41b0-a8d2-1ec87d83ed60 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.024996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00888b40-8f57-48dd-893b-8106cd0d65f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.027322 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00888b40-8f57-48dd-893b-8106cd0d65f0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.109454 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02223b-23e5-41b0-a8d2-1ec87d83ed60 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.131292 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 253/500] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:53.132098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00888b40-8f57-48dd-893b-8106cd0d65f0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.150087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.150766 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.151056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.151337 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.152079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:53.152391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.153815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.170057 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 248/501] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:53.170057 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.234426 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00888b40-8f57-48dd-893b-8106cd0d65f0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.266970 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08167c-27e5-42e6-a9bb-cdf7a1ad0f27 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.269729 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.270920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.271207 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.271485 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.281452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.282920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:53.300793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:53.301913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:53.303517 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:53.303517 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:53.303517 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:53.303517 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:53.303917 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03f4b-03ce-493a-9a31-6f65327bcf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.303917 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03f4b-03ce-493a-9a31-6f65327bcf15 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.304185 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 254/502] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:53.304309 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03f4b-03ce-493a-9a31-6f65327bcf15 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.304309 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03f4b-03ce-493a-9a31-6f65327bcf15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.304555 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.312936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee03f4b-03ce-493a-9a31-6f65327bcf15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9ee03f4b-03ce-493a-9a31-6f65327bcf15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:53.320798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:53.322008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 249/503] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:53.322954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03f4b-03ce-493a-9a31-6f65327bcf15 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.398320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd9dda-26a1-44bd-af16-cc816a5d10d8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.419097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03f4b-03ce-493a-9a31-6f65327bcf15 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.431093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ec559-0992-4b26-adb7-1fb4517e2b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.431885 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ec559-0992-4b26-adb7-1fb4517e2b44 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.432176 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ec559-0992-4b26-adb7-1fb4517e2b44 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.432459 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ec559-0992-4b26-adb7-1fb4517e2b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.442300 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ec559-0992-4b26-adb7-1fb4517e2b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.443765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ec559-0992-4b26-adb7-1fb4517e2b44 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.450577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.451374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.451675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.451955 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] PATH_INFO: `/v3/users/glance` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.459783 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.461266 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-324ec559-0992-4b26-adb7-1fb4517e2b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:53.461372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:53.462331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:53.463323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:53.463323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:53.463323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:53.463323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:53.463323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 255/504] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:53.463830 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9f191-8412-4335-9a85-31f17f03bf94 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.464082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9f191-8412-4335-9a85-31f17f03bf94 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.464409 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9f191-8412-4335-9a85-31f17f03bf94 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.465780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 256/505] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:53.467605 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9f191-8412-4335-9a85-31f17f03bf94 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:41:53.479112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:53.480220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 250/506] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:53.481298 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.569826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9f191-8412-4335-9a85-31f17f03bf94 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.575614 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51220a35-52fe-40db-b753-45c7d6b32d72 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.603044 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.603708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.604001 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.604293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.605026 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:53.605374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.606749 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.608134 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c41a4-a1aa-419f-b58b-b3bdd6329707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.608803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c41a4-a1aa-419f-b58b-b3bdd6329707 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.609122 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c41a4-a1aa-419f-b58b-b3bdd6329707 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.609410 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c41a4-a1aa-419f-b58b-b3bdd6329707 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.610167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c41a4-a1aa-419f-b58b-b3bdd6329707 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:53.610481 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c41a4-a1aa-419f-b58b-b3bdd6329707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.611845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c41a4-a1aa-419f-b58b-b3bdd6329707 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.618174 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 257/507] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:53.618981 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.624002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 251/508] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:53.624002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352993-43cb-4b48-900f-82286bcfca86 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.624002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352993-43cb-4b48-900f-82286bcfca86 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.624002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc352993-43cb-4b48-900f-82286bcfca86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.638413 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-dc352993-43cb-4b48-900f-82286bcfca86 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:53.639606 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-dc352993-43cb-4b48-900f-82286bcfca86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:53.702668 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 252/509] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:53.703544 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc352993-43cb-4b48-900f-82286bcfca86 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.734380 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edbb59-f435-421a-9290-0bd57f0ca9f0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.766325 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.766982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.767271 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.767552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.768270 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:53.768582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.770597 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.781843 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 258/510] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:53.782635 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.801120 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc352993-43cb-4b48-900f-82286bcfca86 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.833920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.834591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.834887 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.835166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.836008 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:53.836322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.837855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.855030 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 253/511] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:53.855849 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.876803 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed91de0-f95e-4292-a16d-e9a5762822c0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.910316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.911142 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.911435 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.911719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:53.919900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:53.921386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:41:53.935872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:53.936937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 259/512] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:53.938052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:53.958151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50ee7a-7320-43a4-937a-1c52f648d533 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:53.991128 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:53.991920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:53.992210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:53.992491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.002270 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.003738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:54.021949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:54.023166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:54.029280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:54.029280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:54.029280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:54.029280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:54.029280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 254/513] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:54.029280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.032564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bec1a-6007-4ec2-a317-46c4cc1d3208 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.065431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.066194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.066483 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.067324 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.077070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.078538 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:54.095683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:54.096676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:54.098059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:54.098059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:54.098059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:54.098059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:54.098059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 260/514] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:54.098059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.121852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3259b551-fc63-468a-94a3-f0adc1dc913c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.153685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.154484 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.154771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.155051 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.161518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.162988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:41:54.177870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:54.178877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 255/515] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:54.179904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.196809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a838d5d-5746-44d0-9a65-d468b8d141b7 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.229813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2d7b4-4916-468e-99ee-8bc5d3f0e3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.230394 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2d7b4-4916-468e-99ee-8bc5d3f0e3dc None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.230678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2d7b4-4916-468e-99ee-8bc5d3f0e3dc None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.230955 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2d7b4-4916-468e-99ee-8bc5d3f0e3dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.273945 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab384b-2a50-433d-932d-3b3161434cf8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.294451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2d7b4-4916-468e-99ee-8bc5d3f0e3dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.295916 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2d7b4-4916-468e-99ee-8bc5d3f0e3dc None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.306367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.307518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.307810 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.308093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.312483 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:54.312483 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.313328 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.331817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 256/516] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:54.333213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.404808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 261/517] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:54.405696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2d7b4-4916-468e-99ee-8bc5d3f0e3dc None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.428857 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a97d6-ecd6-429d-8d88-8f4a60317692 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.461823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.462487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.462773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.463055 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.463777 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:54.464085 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.465514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.477056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 257/518] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:54.477861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.500043 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2d7b4-4916-468e-99ee-8bc5d3f0e3dc None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.532210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.532869 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.533193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.533475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.534198 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:54.534510 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.536391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.547248 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 262/519] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:54.548038 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.574014 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27f95-7bd3-4d6c-974c-51501a620e3f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.611132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.611827 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.612121 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.612406 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.613242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:54.613597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.615558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.632791 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 258/520] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:54.633736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.653977 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ee33d-d935-460f-8ba0-cf78b4d49427 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.686533 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcc1ff-5aa7-46b7-b39a-3f1ae83b213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.687176 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcc1ff-5aa7-46b7-b39a-3f1ae83b213a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.687468 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcc1ff-5aa7-46b7-b39a-3f1ae83b213a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.687749 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcc1ff-5aa7-46b7-b39a-3f1ae83b213a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.727946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c589052-e491-4e61-96e8-2fd7b52090a4 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.751560 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcc1ff-5aa7-46b7-b39a-3f1ae83b213a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.752949 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcc1ff-5aa7-46b7-b39a-3f1ae83b213a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.760433 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.761264 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.761556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.761839 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.772070 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.773551 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:54.791307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:54.792479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:54.793610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:54.793610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:54.793610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:54.793610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:54.793610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 259/521] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:54.793610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.851027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 263/522] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:54.852041 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcc1ff-5aa7-46b7-b39a-3f1ae83b213a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.887876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8bea0-e672-416f-81df-c141c80840ee None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.927448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.928273 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.928568 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.928855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.936006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.937530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:54.947962 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcc1ff-5aa7-46b7-b39a-3f1ae83b213a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:41:54.952079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:54.953330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 260/523] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:54.954400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:54.982547 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb133d-c047-43af-bf16-d63dc94363d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:54.983221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb133d-c047-43af-bf16-d63dc94363d0 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:54.983545 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb133d-c047-43af-bf16-d63dc94363d0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:54.983830 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb133d-c047-43af-bf16-d63dc94363d0 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:54.984543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb133d-c047-43af-bf16-d63dc94363d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:54.986019 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb133d-c047-43af-bf16-d63dc94363d0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.005227 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 264/524] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:41:54 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:41:55.006263 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb133d-c047-43af-bf16-d63dc94363d0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.055185 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77583aa-5588-4fff-9546-3bc60e12ff7b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:55.092205 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:55.092882 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:55.093286 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:55.093578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:55.094483 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=ef8546c698f9455cb81264bcee0d2344) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:55.094808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:55.096316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.103293 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 261/525] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:54 2022] GET /identity/v3/role_assignments?user.id=ef8546c698f9455cb81264bcee0d2344&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:55.104232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.142626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb133d-c047-43af-bf16-d63dc94363d0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:55.181438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:55.182494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:55.183036 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:55.183484 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:55.184505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:55.184993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:55.186806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.203477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 265/526] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:55.204713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.230546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87311209-6f66-49d9-b150-ab4c6bf12aff None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:55.268715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:55.269243 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:55.269538 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:55.269893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:55.286006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:55.287507 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:55.306864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:55.307953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:55.308968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:55.308968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:55.308968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:55.308968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:55.308968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 262/527] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:55.314000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.324416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbef1e-aff0-47c5-8f78-a20d428fe0c3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:55.367262 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:55.368726 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:55.369281 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:55.370106 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:55.371506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:55.372062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:55.374538 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.421596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 266/528] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:55.422407 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.436910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689457a-dad5-4a56-a0f6-31f7cc6b0906 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:55.477655 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:55.478571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:55.478937 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:55.479301 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:55.486780 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:55.488400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:41:55.505019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:55.506131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:55.507158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 263/529] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:55.507781 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.542511 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16194dfe-a674-4e45-93fb-6cadffa963ff None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:55.587096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:55.588380 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:55.589153 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:55.589626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:55.590700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:55.591178 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:55.593438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.609126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 267/530] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:55.610229 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.610891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efa5e9-2d52-47a4-9eb2-5715d1ac3990 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:55.661070 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bdac6-b783-4bad-9240-097135a9e7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:55.662035 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdac6-b783-4bad-9240-097135a9e7a9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:55.662486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdac6-b783-4bad-9240-097135a9e7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:55.663419 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bdac6-b783-4bad-9240-097135a9e7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:55.743195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bdac6-b783-4bad-9240-097135a9e7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:55.745250 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bdac6-b783-4bad-9240-097135a9e7a9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.762633 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2da33f-4d64-4d4d-bce5-fa30fd9e7c1f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:55.820606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:55.821579 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:55.822044 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:55.822509 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:55.823464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:55.824050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:55.825914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:55.851994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 268/531] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:55.853113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.886858 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 264/532] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:55.888422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bdac6-b783-4bad-9240-097135a9e7a9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:55.971221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591d4c1-a7c5-4d92-9b39-0b01a8413937 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:56.004355 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bdac6-b783-4bad-9240-097135a9e7a9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:56.010268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:56.011321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:56.011857 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:56.012348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:56.019571 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:56.021497 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:41:56.036900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:56.038066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:56.039120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:41:56.039120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:41:56.039120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:41:56.039120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:41:56.039120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:41:56.039120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:41:56.039120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:41:56.039120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:56.041278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 269/533] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:56.042268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:56.046020 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:56.046803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:56.047176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:56.047539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:56.048480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=4ff3fa04f3f749debec784ed5d2818a5) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:56.048986 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:56.051954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:56.061203 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 265/534] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:55 2022] GET /identity/v3/role_assignments?user.id=4ff3fa04f3f749debec784ed5d2818a5&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:56.062370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:56.233763 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd40ca-97a1-4b6a-9196-e23f15014740 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:56.278289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:56.279538 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:56.280087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:56.280705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:56.280990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927f531-c28c-4aad-8445-5b1683ccf69f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:56.295911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:56.297760 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:56.315841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:56.316979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:56.318039 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:56.318039 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:56.318039 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:56.318039 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:56.320455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 270/535] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:56.325371 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:56.344626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:56.345349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:56.345646 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:56.345933 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:56.346750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:56.347077 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:56.349855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:56.371916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 266/536] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:56.377393 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:56.439191 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003a106-f53a-4973-bdb0-b29ae50bca95 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:56.475411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:56.476477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:56.476980 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:56.477500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:56.478440 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:56.478980 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:56.480613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:56.493655 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 271/537] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:56.498496 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:56.536798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a1a8e-4bbd-40c5-b880-be1dbbc87eb2 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:56.588195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:56.589305 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:56.589829 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:56.590623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:56.592686 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=8c4f76337b10446999b1e9eef4495ef7) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:56.593390 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:56.595936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:56.612147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 267/538] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:56 2022] GET /identity/v3/role_assignments?user.id=8c4f76337b10446999b1e9eef4495ef7&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:56.671212 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bdad3a-827e-4e15-b8d8-be47c9f07670 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:56.708502 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:56.709720 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:56.710259 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:56.710766 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:56.718634 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:56.720436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:41:56.735950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:56.737021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:56.738098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:41:56.738098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:41:56.738098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:41:56.738098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:41:56.738098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:41:56.738098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:41:56.738098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:41:56.738098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:56.740891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 272/539] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:56.746225 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:56.906131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e4532-841e-4b4c-bb4f-eef7f252cc26 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:56.961851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cbc84-8ef8-40d8-b9b4-7b414d09a238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:56.962544 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cbc84-8ef8-40d8-b9b4-7b414d09a238 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:56.962830 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cbc84-8ef8-40d8-b9b4-7b414d09a238 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:56.963119 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cbc84-8ef8-40d8-b9b4-7b414d09a238 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:56.963934 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cbc84-8ef8-40d8-b9b4-7b414d09a238 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:56.964265 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cbc84-8ef8-40d8-b9b4-7b414d09a238 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:56.965778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cbc84-8ef8-40d8-b9b4-7b414d09a238 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:56.984517 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 268/540] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:56.990488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:57.137846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dc619-383e-4b31-a379-987f326f149b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:57.195160 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79975330-197f-4bda-b34e-03369a5707fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:57.195837 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79975330-197f-4bda-b34e-03369a5707fb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:57.196134 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79975330-197f-4bda-b34e-03369a5707fb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:57.196425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79975330-197f-4bda-b34e-03369a5707fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:57.197323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79975330-197f-4bda-b34e-03369a5707fb None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=88be715120ee4267a4a206b71fa21224) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:57.197649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79975330-197f-4bda-b34e-03369a5707fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:57.199116 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79975330-197f-4bda-b34e-03369a5707fb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:57.209673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 273/541] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:41:56 2022] GET /identity/v3/role_assignments?user.id=88be715120ee4267a4a206b71fa21224&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:57.605741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7285f35-cb18-4a16-9522-be33ebfc6101 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:57.606314 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7285f35-cb18-4a16-9522-be33ebfc6101 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:57.606314 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7285f35-cb18-4a16-9522-be33ebfc6101 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:57.607953 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 269/542] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:57.614302 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78400c-5aab-4f19-881f-1bf9a1394840 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:57.614800 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78400c-5aab-4f19-881f-1bf9a1394840 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:57.614899 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78400c-5aab-4f19-881f-1bf9a1394840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:57.635385 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-ae78400c-5aab-4f19-881f-1bf9a1394840 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:57.636591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-ae78400c-5aab-4f19-881f-1bf9a1394840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:57.706276 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 274/543] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:57.730223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7285f35-cb18-4a16-9522-be33ebfc6101 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:57.789217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9388e-f32f-4d0e-87eb-c404adc349f6 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:57.789542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9388e-f32f-4d0e-87eb-c404adc349f6 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:57.789791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9388e-f32f-4d0e-87eb-c404adc349f6 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:57.791167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 275/544] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:57.796810 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531929-af86-4e27-a029-48329e77c1a1 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:57.797125 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531929-af86-4e27-a029-48329e77c1a1 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:57.797413 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531929-af86-4e27-a029-48329e77c1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:57.817953 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-4f531929-af86-4e27-a029-48329e77c1a1 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:57.819153 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-4f531929-af86-4e27-a029-48329e77c1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:57.886810 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7285f35-cb18-4a16-9522-be33ebfc6101 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:57.928050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 276/545] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:41:57.963393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531929-af86-4e27-a029-48329e77c1a1 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:57.963616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ea2f-4e52-4474-a1eb-8246d5191854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:57.964405 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ea2f-4e52-4474-a1eb-8246d5191854 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:57.964698 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ea2f-4e52-4474-a1eb-8246d5191854 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:57.964984 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ea2f-4e52-4474-a1eb-8246d5191854 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:57.978108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ea2f-4e52-4474-a1eb-8246d5191854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-b5b5ea2f-4e52-4474-a1eb-8246d5191854 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:57.986174 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:57.987245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:57.988189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 270/546] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:57.996812 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ea2f-4e52-4474-a1eb-8246d5191854 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:58.131476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ea2f-4e52-4474-a1eb-8246d5191854 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:58.164104 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531929-af86-4e27-a029-48329e77c1a1 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:58.174559 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:58.175593 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.176132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.176622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.177667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:58.178516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:58.182010 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:58.204571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 271/547] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:41:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:58.205876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:58.232575 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e8406-8ee7-46da-86b2-bdefad1fc0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:58.234022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e8406-8ee7-46da-86b2-bdefad1fc0b8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.236017 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e8406-8ee7-46da-86b2-bdefad1fc0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.236538 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e8406-8ee7-46da-86b2-bdefad1fc0b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.256392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e8406-8ee7-46da-86b2-bdefad1fc0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-fd3e8406-8ee7-46da-86b2-bdefad1fc0b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:58.271863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:58.272924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:58.273919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:58.279319 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 277/548] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:41:58.285023 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e8406-8ee7-46da-86b2-bdefad1fc0b8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:58.362458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6b662-9590-4171-8c4b-2d8357303aa2 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:58.407815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:58.408634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.408923 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.409235 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.434276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:58.436106 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:58.469063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e8406-8ee7-46da-86b2-bdefad1fc0b8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:58.470031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:58.471021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:58.472007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:58.472007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:58.472007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:58.472007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:58.472007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 272/549] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:58.477930 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:58.503063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:58.503063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.503063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.503063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.503524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:58.503524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:58.508834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:58.524077 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 278/550] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:41:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:58.529653 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:58.653064 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03160d8d-7b30-4179-95c9-79299e98dcb4 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:58.686952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91232fdd-c476-40f2-8478-d419c7c9b89a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:58.699150 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:58.699978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.700254 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.700540 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.723560 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:58.724018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:58.724850 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.725394 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.725609 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:58.726097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.727041 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:58.727564 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:58.729259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:58.742604 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 273/551] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:58.749964 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:58.752722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:58.753765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:58.755332 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:58.755332 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:58.755332 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:58.755332 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:58.766725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 279/552] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:58.772637 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:58.875612 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19e759-8e5f-4bb9-b74b-a509f163cda6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:58.888737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24465e36-6f51-4f6d-8060-c34104c2d28b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:58.920358 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c541a-e7eb-4830-ad58-f899eba97ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:58.921475 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c541a-e7eb-4830-ad58-f899eba97ad1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.921976 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c541a-e7eb-4830-ad58-f899eba97ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.922465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c541a-e7eb-4830-ad58-f899eba97ad1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.925922 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5afe1d-ab94-45e8-b44a-bb85866686a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:58.927021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5afe1d-ab94-45e8-b44a-bb85866686a6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.927317 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5afe1d-ab94-45e8-b44a-bb85866686a6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.927603 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5afe1d-ab94-45e8-b44a-bb85866686a6 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.928345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5afe1d-ab94-45e8-b44a-bb85866686a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:58.928661 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5afe1d-ab94-45e8-b44a-bb85866686a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:58.930103 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5afe1d-ab94-45e8-b44a-bb85866686a6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:58.932598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c541a-e7eb-4830-ad58-f899eba97ad1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:58.935064 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c541a-e7eb-4830-ad58-f899eba97ad1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:58.946400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 280/553] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:41:58.947769 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51b130-78ba-480f-b027-601fa0cd1bb8 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.948071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51b130-78ba-480f-b027-601fa0cd1bb8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.950108 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51b130-78ba-480f-b027-601fa0cd1bb8 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.951789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 281/554] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:58.952531 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51b130-78ba-480f-b027-601fa0cd1bb8 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-710c541a-e7eb-4830-ad58-f899eba97ad1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:41:58.953798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:58.954822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:41:58.955889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:58.958658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 274/555] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:41:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:58.960164 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d3951-478c-4bcf-9a04-6ced943c32ab None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:58.960710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d3951-478c-4bcf-9a04-6ced943c32ab None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:58.961258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d3951-478c-4bcf-9a04-6ced943c32ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:58.979217 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-1a1d3951-478c-4bcf-9a04-6ced943c32ab None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:58.980796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-1a1d3951-478c-4bcf-9a04-6ced943c32ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:59.048814 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 275/556] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:59.050100 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d3951-478c-4bcf-9a04-6ced943c32ab None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.062691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51b130-78ba-480f-b027-601fa0cd1bb8 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.106396 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.106997 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.107291 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.107574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.116244 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.117925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:41:59.134082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:59.135345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:59.136532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 282/557] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:41:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:59.138364 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.163736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d3951-478c-4bcf-9a04-6ced943c32ab None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.200105 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.201192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.201707 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.202215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.203287 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:59.203825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.205589 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.224554 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 276/558] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:59.225807 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.242322 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f99f5d-13bd-4122-8842-be034af6d1fc None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.277803 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9495982-aaa3-4da2-8c6e-19ad6131a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.278655 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9495982-aaa3-4da2-8c6e-19ad6131a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.278942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9495982-aaa3-4da2-8c6e-19ad6131a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.279226 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9495982-aaa3-4da2-8c6e-19ad6131a9a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.290733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9495982-aaa3-4da2-8c6e-19ad6131a9a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-f9495982-aaa3-4da2-8c6e-19ad6131a9a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:41:59.298699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:59.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:59.300786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 283/559] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:59.301714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9495982-aaa3-4da2-8c6e-19ad6131a9a0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.333946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49da4a-7c3c-4c06-8a7e-cc8bbd2ae74e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.368187 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.368862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.369181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.369464 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.370310 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:59.370625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.372098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.392705 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 277/560] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:59.393980 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.409634 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9495982-aaa3-4da2-8c6e-19ad6131a9a0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.443038 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83939d1-3d07-48aa-836b-9fec9e9f331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.443700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83939d1-3d07-48aa-836b-9fec9e9f331a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.443986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83939d1-3d07-48aa-836b-9fec9e9f331a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.444264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83939d1-3d07-48aa-836b-9fec9e9f331a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.490623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5abb98-18d8-4aa8-b312-7c6f618ee52b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.510988 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83939d1-3d07-48aa-836b-9fec9e9f331a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.512670 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83939d1-3d07-48aa-836b-9fec9e9f331a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.527179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc28170-f73f-47e6-b358-603f40d25626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.527865 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc28170-f73f-47e6-b358-603f40d25626 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.528159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc28170-f73f-47e6-b358-603f40d25626 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.528447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc28170-f73f-47e6-b358-603f40d25626 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.529252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc28170-f73f-47e6-b358-603f40d25626 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:59.529570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc28170-f73f-47e6-b358-603f40d25626 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.530944 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc28170-f73f-47e6-b358-603f40d25626 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.543756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 278/561] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:59.543756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c08fe-bebf-41c7-86c0-dac7765e1984 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.543756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c08fe-bebf-41c7-86c0-dac7765e1984 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.543756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c08fe-bebf-41c7-86c0-dac7765e1984 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.543756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 279/562] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:41:59.543756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c08fe-bebf-41c7-86c0-dac7765e1984 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.620970 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 284/563] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:41:59.621948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83939d1-3d07-48aa-836b-9fec9e9f331a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.644641 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c08fe-bebf-41c7-86c0-dac7765e1984 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.678238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8b48e-ce20-48c5-ad85-0f5dcfe7adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.679382 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b48e-ce20-48c5-ad85-0f5dcfe7adb3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.679871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b48e-ce20-48c5-ad85-0f5dcfe7adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.680349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b48e-ce20-48c5-ad85-0f5dcfe7adb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.691172 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8b48e-ce20-48c5-ad85-0f5dcfe7adb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.693867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8b48e-ce20-48c5-ad85-0f5dcfe7adb3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-46e8b48e-ce20-48c5-ad85-0f5dcfe7adb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:59.711666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:59.712710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:59.713757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:59.713757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:59.713757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:59.713757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:41:59.713757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 280/564] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:59.714546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80031460-6218-4793-9998-73deef21ce1d None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.714546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80031460-6218-4793-9998-73deef21ce1d None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.714806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80031460-6218-4793-9998-73deef21ce1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.728744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83939d1-3d07-48aa-836b-9fec9e9f331a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.731550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-80031460-6218-4793-9998-73deef21ce1d None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:41:59.732763 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-80031460-6218-4793-9998-73deef21ce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:41:59.765808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.766554 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.766902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.767132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.778971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.780468 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:59.799080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:59.800278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:59.801496 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:59.801496 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:59.801496 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:59.801496 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:59.803065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 285/565] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:59.813145 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 281/566] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:41:59.813145 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80031460-6218-4793-9998-73deef21ce1d None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.813387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.914745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80031460-6218-4793-9998-73deef21ce1d None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.922373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e0536-4d87-4127-91e1-b50ad4dacd19 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:41:59.951257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.951935 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.952228 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.952514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.953274 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:41:59.953587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.954972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.956157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:41:59.956957 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:41:59.957840 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:41:59.958129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:41:59.966496 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 282/567] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:41:59.967286 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:41:59.968383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:41:59.969902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:41:59.987644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:41:59.989405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:41:59.990697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:41:59.990697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:41:59.990697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:41:59.990697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:41:59.990697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 286/568] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:41:59.990697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.065430 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20dfcc1-439d-43a9-b991-a36e33522e1d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.083971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff24cfd-5020-4a35-a089-8ceb0792ca36 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.097795 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de4a0e-e3ac-4812-9c0b-59be4a7d04ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.098468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4a0e-e3ac-4812-9c0b-59be4a7d04ad None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.098760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4a0e-e3ac-4812-9c0b-59be4a7d04ad None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.099047 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4a0e-e3ac-4812-9c0b-59be4a7d04ad None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.099780 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de4a0e-e3ac-4812-9c0b-59be4a7d04ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:00.100096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de4a0e-e3ac-4812-9c0b-59be4a7d04ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.101498 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de4a0e-e3ac-4812-9c0b-59be4a7d04ad None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.112763 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 283/569] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:00.114565 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441b76a-7a59-4d8a-9071-913dd3d32120 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.114863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441b76a-7a59-4d8a-9071-913dd3d32120 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.115151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441b76a-7a59-4d8a-9071-913dd3d32120 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.116400 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 284/570] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:00.117132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441b76a-7a59-4d8a-9071-913dd3d32120 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.120895 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071f25f-6850-490a-96b4-8c2cbf15dab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.121732 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071f25f-6850-490a-96b4-8c2cbf15dab1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.122018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071f25f-6850-490a-96b4-8c2cbf15dab1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.122299 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071f25f-6850-490a-96b4-8c2cbf15dab1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.132993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071f25f-6850-490a-96b4-8c2cbf15dab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b071f25f-6850-490a-96b4-8c2cbf15dab1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:00.140733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:00.141808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:00.142740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 287/571] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:00.143686 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071f25f-6850-490a-96b4-8c2cbf15dab1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.219439 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441b76a-7a59-4d8a-9071-913dd3d32120 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.240102 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071f25f-6850-490a-96b4-8c2cbf15dab1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.251980 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b3f8f-0a68-4fd0-bdbe-6d2e29f6a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.252778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b3f8f-0a68-4fd0-bdbe-6d2e29f6a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.253093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b3f8f-0a68-4fd0-bdbe-6d2e29f6a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.253388 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b3f8f-0a68-4fd0-bdbe-6d2e29f6a1f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.260561 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b3f8f-0a68-4fd0-bdbe-6d2e29f6a1f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.262029 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b3f8f-0a68-4fd0-bdbe-6d2e29f6a1f2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.272379 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.273032 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.273340 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.273668 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.274348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:00.274663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-e83b3f8f-0a68-4fd0-bdbe-6d2e29f6a1f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:00.275718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:00.276743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:00.277802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:00.277802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:00.277802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:00.277802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:00.277802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:00.277802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:00.277802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:00.277802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:00.278403 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.278486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 285/572] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:00.278486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc1b05-0212-46b3-993b-d816223db712 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.278486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc1b05-0212-46b3-993b-d816223db712 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.278698 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc1b05-0212-46b3-993b-d816223db712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.292090 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 288/573] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:00.292090 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.294476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-c1dc1b05-0212-46b3-993b-d816223db712 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:00.295684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-c1dc1b05-0212-46b3-993b-d816223db712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:00.359489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 286/574] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:00.360252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc1b05-0212-46b3-993b-d816223db712 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.385414 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b46cea-c772-4191-a8c1-d76c74316c77 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.418478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.419259 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.419551 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.419833 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] PATH_INFO: `/v3/users/glance` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.427416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.428878 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:00.443192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:00.444417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 289/575] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:00.445647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.466474 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc1b05-0212-46b3-993b-d816223db712 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.499691 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.500372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.500665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.500949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.501765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:00.502082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.503458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.511999 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 287/576] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:00.512653 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.541717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a615f-e5cd-49df-9482-859901013b16 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.574812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.575491 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.575784 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.576067 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.576870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:00.577229 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.578682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.589872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 290/577] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:00.590662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.616321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210dcfc-81b0-438f-903f-44c2990af96a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.647694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.648493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.648783 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.649092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.655567 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.657021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:00.671572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:00.672609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:00.673648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 288/578] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:00.674195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.688188 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd6821-1897-492c-8168-ddb09f4fb75c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.719691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59179155-fb2d-4349-a82d-7a0b9c9ea301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.720482 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59179155-fb2d-4349-a82d-7a0b9c9ea301 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.720772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59179155-fb2d-4349-a82d-7a0b9c9ea301 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.721081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59179155-fb2d-4349-a82d-7a0b9c9ea301 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.731636 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59179155-fb2d-4349-a82d-7a0b9c9ea301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-59179155-fb2d-4349-a82d-7a0b9c9ea301 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:00.739923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:00.741118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:00.742223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 291/579] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:00.743012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59179155-fb2d-4349-a82d-7a0b9c9ea301 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.774255 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e74bf-6cae-4111-b7c5-d469ebf1b2f8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.806776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.807543 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.807906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.808269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.809228 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:00.809617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.811118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.827524 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 289/580] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:00.828392 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.836695 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59179155-fb2d-4349-a82d-7a0b9c9ea301 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.870210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.870938 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.871228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.871509 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.881879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.883322 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:00.900297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:00.901526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:00.902724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:00.902724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:00.902724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:00.902724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:00.903124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 292/581] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:00.904093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:00.924593 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067ee8f-790f-4136-84a7-00121d05a88f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:00.956460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92890340-d4df-4fcd-b781-0f60cb9b8828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:00.957920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92890340-d4df-4fcd-b781-0f60cb9b8828 None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:00.958209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92890340-d4df-4fcd-b781-0f60cb9b8828 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:00.958490 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92890340-d4df-4fcd-b781-0f60cb9b8828 None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/ef8546c698f9455cb81264bcee0d2344/roles/d28cdc73a0fc42f6b6e0c45768d418ae` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:00.972445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92890340-d4df-4fcd-b781-0f60cb9b8828 None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=ef8546c698f9455cb81264bcee0d2344, role_id=d28cdc73a0fc42f6b6e0c45768d418ae)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:00.974486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92890340-d4df-4fcd-b781-0f60cb9b8828 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:00.986889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 290/582] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:00 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/ef8546c698f9455cb81264bcee0d2344/roles/d28cdc73a0fc42f6b6e0c45768d418ae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:42:00.987698 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92890340-d4df-4fcd-b781-0f60cb9b8828 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.001415 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762d9c2-02b6-4ab0-83b1-0d24ce05ec29 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.037805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.038488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.038781 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.039070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.039886 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:01.040207 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.041726 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.054842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 293/583] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:01.055555 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.103534 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92890340-d4df-4fcd-b781-0f60cb9b8828 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.138775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.139386 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.139678 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.139969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.140738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:01.141072 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.142485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.156376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 291/584] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:01.156376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.176301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cde7f-d2f5-44aa-9541-fc8456e9fea5 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.211211 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.212012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.212306 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.212586 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.223610 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.225087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:01.242827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:01.244129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:01.245249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:01.245249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:01.245249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:01.245249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:01.245249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 294/585] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:01.245249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.267102 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588caeb4-6a55-4139-b3ca-d0e594485a69 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.301046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.301731 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.302019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.302302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.303035 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:01.303347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.304724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.322687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 292/586] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:01.322687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.344817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae81531-141b-4e54-b8d5-912f9de7aa34 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.383317 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.383983 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.384267 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.384539 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.385418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=48c22f093f45490e996d673b027bf391) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:01.385740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.387172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.401158 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 295/587] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/role_assignments?user.id=48c22f093f45490e996d673b027bf391&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:01.401158 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.438827 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686134d-4e54-44ad-affb-fc914c69ff71 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.476023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.477145 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.477595 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.477976 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.489897 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.491625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.503330 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea2e89-ee69-4ad4-bd1e-823584308d2b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:01.513259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:01.514365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:01.515415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:01.515415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:01.515415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:01.515415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:01.515415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 293/588] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:01.516164 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.545286 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.546369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.546877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.547366 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.548323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:01.548891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.550577 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.564572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 296/589] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:01.566173 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.640589 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a30759-3a20-4254-bb2b-f8c73cb47f1f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.673761 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a506b-be85-4b39-9412-b0e2d43c44a3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.675531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.676776 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.677076 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.677363 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.684806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.686320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:01.702234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:01.703411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:42:01.704525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:01.708522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 294/590] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:01.709542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.717421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.718445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.718962 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.719458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.720429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:01.720966 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.722987 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.736065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 297/591] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:01.737802 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.815465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb14a8f-6329-4e7c-8e9f-928c040b6e5b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.841397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef4fd-8900-4532-b3ad-1585bcafb00a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:01.850294 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.851111 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.851397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.851673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.858707 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.860165 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:01.874370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:01.875397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:01.876408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 295/592] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:01.877007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:01.884379 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:01.885076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:01.885369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:01.885648 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:01.886506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:01.886818 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:01.888243 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:01.915070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 298/593] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:01.915980 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.003723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56925-31c7-49ea-a5ff-203040753e05 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.040228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e257ac-4b96-4590-9b4c-9fe0fd9ff53c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.046237 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.047060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.047351 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.047635 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.057811 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.059684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:02.078009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:02.079084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 296/594] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:02.080177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.084960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.085666 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.085972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.086224 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.087031 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:02.087345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.088790 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.102432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 299/595] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:02.103261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.208202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152c5c8-c63a-4527-8cdc-e1e047b1f3b7 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.215758 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe003b2c-2311-4235-8c84-12100eaa6fba None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.250256 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.251096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.251384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.251730 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.253247 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:02.253585 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.255285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.258772 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.259865 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.260388 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.260894 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.274788 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.276622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.280127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 300/596] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:02.281212 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:02.302550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:02.303626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:02.304631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:02.304631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:02.304631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:02.304631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:02.307399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 297/597] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:02.308450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.421473 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c65702-5bec-4739-8100-3453e52192f1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.439220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b7283-b97f-4f58-9ff6-564009670c18 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.466973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0de00-4498-44b9-a0b4-5c23bb2f1f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.468092 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0de00-4498-44b9-a0b4-5c23bb2f1f06 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.468490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0de00-4498-44b9-a0b4-5c23bb2f1f06 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.468909 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0de00-4498-44b9-a0b4-5c23bb2f1f06 None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/4ff3fa04f3f749debec784ed5d2818a5/roles/d28cdc73a0fc42f6b6e0c45768d418ae` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.475113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.476104 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.476580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.477015 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.487507 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0de00-4498-44b9-a0b4-5c23bb2f1f06 None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=4ff3fa04f3f749debec784ed5d2818a5, role_id=d28cdc73a0fc42f6b6e0c45768d418ae)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.488164 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.489408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0de00-4498-44b9-a0b4-5c23bb2f1f06 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.490104 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.507990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 301/598] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:02 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/4ff3fa04f3f749debec784ed5d2818a5/roles/d28cdc73a0fc42f6b6e0c45768d418ae => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:42:02.508208 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0de00-4498-44b9-a0b4-5c23bb2f1f06 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:02.514321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:02.515385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:02.516591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:02.516591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:02.516591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:02.516591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:02.524156 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 298/599] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:02.525497 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.640406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0de00-4498-44b9-a0b4-5c23bb2f1f06 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.686347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.687361 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.687718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.688071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.688900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:02.689376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.691016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.703903 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e982fc-c5f7-4ad2-bd40-35930a8ab55f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.705642 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 302/600] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:02.713688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.742115 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.743126 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.743618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.744095 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.745028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:02.745581 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.747166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.760940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 299/601] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:02.767964 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.827540 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066be7d-ff73-4b7c-869d-16cc440dde87 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.870113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.870927 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.871217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.871501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.880033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.881542 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:02.898083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:02.899194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:02.900228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:02.900228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:02.900228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:02.900228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:02.900228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:02.900228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:02.900228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:02.900228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:02.902527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c96d0-8887-4389-8c0b-d6df138f5a7c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:02.903936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 303/602] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:02.905164 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:02.948077 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:02.949743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:02.950274 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:02.950841 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:02.961166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:02.963478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:02.982823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:02.983942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:02.984958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:02.984958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:02.984958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:02.984958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:02.984958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:02.984958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:02.984958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:02.984958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:02.987715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 300/603] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:02.994132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:03.019277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1224c-e739-42d7-be27-ed1ff577c2d3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:03.053978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:03.054645 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.054936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.055220 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.056020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:03.056335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:03.057863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:03.075188 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 304/604] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:03.076419 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:03.103413 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1cad-ba6d-492c-8138-88dcb1c54b74 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:03.139309 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:03.140372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.140952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.141485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.142565 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:03.143107 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:03.144927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:03.158130 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 301/605] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:03.164155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:03.178318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053bbd3-3fbc-469f-861e-b73fcf5429a0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:03.213025 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff6861-ea69-4a82-85d6-1e04b3c4164e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:03.214170 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff6861-ea69-4a82-85d6-1e04b3c4164e None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.214329 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff6861-ea69-4a82-85d6-1e04b3c4164e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.214531 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff6861-ea69-4a82-85d6-1e04b3c4164e None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/8c4f76337b10446999b1e9eef4495ef7/roles/d28cdc73a0fc42f6b6e0c45768d418ae` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.229864 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff6861-ea69-4a82-85d6-1e04b3c4164e None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=8c4f76337b10446999b1e9eef4495ef7, role_id=d28cdc73a0fc42f6b6e0c45768d418ae)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:03.231320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff6861-ea69-4a82-85d6-1e04b3c4164e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:03.253587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 305/606] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:03 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/8c4f76337b10446999b1e9eef4495ef7/roles/d28cdc73a0fc42f6b6e0c45768d418ae => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:42:03.275469 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0b039-2e2d-45bf-a2f0-a1bbdf00f556 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:03.315242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76517ca5-3606-4154-afd0-8b786a253a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:03.317191 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517ca5-3606-4154-afd0-8b786a253a9c None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.317707 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517ca5-3606-4154-afd0-8b786a253a9c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.318253 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76517ca5-3606-4154-afd0-8b786a253a9c None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/88be715120ee4267a4a206b71fa21224/roles/d28cdc73a0fc42f6b6e0c45768d418ae` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.336522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76517ca5-3606-4154-afd0-8b786a253a9c None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=88be715120ee4267a4a206b71fa21224, role_id=d28cdc73a0fc42f6b6e0c45768d418ae)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:03.338473 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76517ca5-3606-4154-afd0-8b786a253a9c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:03.361311 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 302/607] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:03 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/88be715120ee4267a4a206b71fa21224/roles/d28cdc73a0fc42f6b6e0c45768d418ae => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:42:03.596767 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dedaa-1be1-4f75-b833-fa6626549d09 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.596986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dedaa-1be1-4f75-b833-fa6626549d09 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.598079 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dedaa-1be1-4f75-b833-fa6626549d09 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.599695 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 306/608] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:03.605309 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccdd85-6d61-40a0-b067-3db242a32177 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.605873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccdd85-6d61-40a0-b067-3db242a32177 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.606378 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccdd85-6d61-40a0-b067-3db242a32177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.624064 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-94ccdd85-6d61-40a0-b067-3db242a32177 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:03.625596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-94ccdd85-6d61-40a0-b067-3db242a32177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:03.699557 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 303/609] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:03.724697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382dedaa-1be1-4f75-b833-fa6626549d09 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:03.861999 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382dedaa-1be1-4f75-b833-fa6626549d09 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:03.911190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2958b-4b8d-46a3-bd56-2ecd2b990df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:03.912011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2958b-4b8d-46a3-bd56-2ecd2b990df4 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.912302 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2958b-4b8d-46a3-bd56-2ecd2b990df4 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.912586 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2958b-4b8d-46a3-bd56-2ecd2b990df4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.925063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2958b-4b8d-46a3-bd56-2ecd2b990df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-eaa2958b-4b8d-46a3-bd56-2ecd2b990df4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:03.932838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:03.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:03.934925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 307/610] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:03.940729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccdd85-6d61-40a0-b067-3db242a32177 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:03.953115 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8540-9550-4184-bc26-441a5c2d2bcf None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.953115 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8540-9550-4184-bc26-441a5c2d2bcf None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.953115 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd8540-9550-4184-bc26-441a5c2d2bcf None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.954777 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 308/611] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:03.962590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e044-d2f2-425a-9ebb-0f6aeecef4ee None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:03.962948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e044-d2f2-425a-9ebb-0f6aeecef4ee None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:03.963300 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2e044-d2f2-425a-9ebb-0f6aeecef4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:03.982791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-cbc2e044-d2f2-425a-9ebb-0f6aeecef4ee None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:03.984081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-cbc2e044-d2f2-425a-9ebb-0f6aeecef4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:04.072858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 309/612] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:42:04.111256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccdd85-6d61-40a0-b067-3db242a32177 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:04.113026 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2e044-d2f2-425a-9ebb-0f6aeecef4ee None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:04.178379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:04.179357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:04.179825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:04.180288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:04.181256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:04.181730 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:04.183302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:04.202836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 304/613] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:04.221402 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:04.246438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2e044-d2f2-425a-9ebb-0f6aeecef4ee None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:04.288738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98c5f5-4aff-4400-b67e-e20453a03257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:04.289582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c5f5-4aff-4400-b67e-e20453a03257 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:04.289872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c5f5-4aff-4400-b67e-e20453a03257 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:04.290154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c5f5-4aff-4400-b67e-e20453a03257 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:04.314193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98c5f5-4aff-4400-b67e-e20453a03257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-1d98c5f5-4aff-4400-b67e-e20453a03257 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:04.322556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:04.323631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:04.324584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:04.337159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 310/614] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:04.338091 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98c5f5-4aff-4400-b67e-e20453a03257 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:04.395683 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5961937-1107-4d29-9eff-110732a2244d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:04.470668 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:04.471484 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:04.471771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:04.472054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:04.472726 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98c5f5-4aff-4400-b67e-e20453a03257 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:04.497557 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:04.499104 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:04.513089 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:04.515348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:04.515633 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:04.516845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:04.518016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:04.518338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:04.519945 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:04.530327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:04.532512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:04.534858 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:04.534858 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:04.534858 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:04.534858 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:04.540126 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 305/615] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:04.540126 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:04.540763 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 311/616] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:04.544195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:04.668920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e2951-df87-4440-bba9-f74d149a089b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:04.727303 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:04.728039 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:04.728328 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:04.728625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:04.729419 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:04.729730 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:04.731161 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:04.743856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 306/617] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:04.750381 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e79f0c-ac1e-4111-a60c-09a8ed6404c7 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:04.754658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:04.798554 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:04.799360 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:04.799644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:04.799919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:04.826109 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:04.827574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:04.854574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:04.855626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:04.856641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:04.856641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:04.856641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:04.856641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:04.858614 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 312/618] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:04.867516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:04.928711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00a36b-8dca-4e40-8983-c613bd8c6a43 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:04.998163 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:04.999683 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.000188 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.000638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] PATH_INFO: `/v3/users/nova` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.018744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:05.020607 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:05.045465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:05.046804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:05.048060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:05.056067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 307/619] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:42:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:05.057583 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:05.111207 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f497bbf-1c2c-4a30-8d2b-f65dc3e6de54 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:05.164616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c75d2e-8278-4fbc-b201-f52b7d6b09df None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:05.201172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:05.202528 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.203289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.204214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.205155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:05.206103 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:05.206955 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.207434 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:05.207875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.208301 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.210374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:05.211265 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:05.211599 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:05.214697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:05.243933 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 313/620] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:05.255602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 308/621] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:42:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:05.258272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:05.269990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:05.414372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dda31-ec5b-4a9e-9aaf-98f4e996b548 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:05.449161 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4926-f69d-49c7-8ea5-9878ad38d784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:05.450262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4926-f69d-49c7-8ea5-9878ad38d784 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.450711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4926-f69d-49c7-8ea5-9878ad38d784 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.451241 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4926-f69d-49c7-8ea5-9878ad38d784 None admin] PATH_INFO: `/v3/users/swift` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.458826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213f90d-962a-4d2f-ae8e-89271ba8271a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:05.459245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4926-f69d-49c7-8ea5-9878ad38d784 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:05.460694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4926-f69d-49c7-8ea5-9878ad38d784 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-de0e4926-f69d-49c7-8ea5-9878ad38d784 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:05.476261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:05.477448 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:05.478539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:05.482185 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 309/622] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:05.483551 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e861-eddc-486b-a206-0e98cab95721 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.483846 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e861-eddc-486b-a206-0e98cab95721 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.484120 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e861-eddc-486b-a206-0e98cab95721 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.485959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 310/623] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:05.490142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47e861-eddc-486b-a206-0e98cab95721 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:05.506206 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0ae0-3810-4c3d-b4fc-55aeeed29e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:05.506918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0ae0-3810-4c3d-b4fc-55aeeed29e2f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.507139 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0ae0-3810-4c3d-b4fc-55aeeed29e2f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.507415 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0ae0-3810-4c3d-b4fc-55aeeed29e2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.589685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f0ae0-3810-4c3d-b4fc-55aeeed29e2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:05.591552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f0ae0-3810-4c3d-b4fc-55aeeed29e2f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:05.609854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47e861-eddc-486b-a206-0e98cab95721 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:05.656599 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4670cb-8a20-4acc-a13e-888fbf43a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:05.657708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4670cb-8a20-4acc-a13e-888fbf43a60a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.658210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4670cb-8a20-4acc-a13e-888fbf43a60a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.658736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4670cb-8a20-4acc-a13e-888fbf43a60a None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.659873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4670cb-8a20-4acc-a13e-888fbf43a60a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:05.660455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4670cb-8a20-4acc-a13e-888fbf43a60a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:05.662155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4670cb-8a20-4acc-a13e-888fbf43a60a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:05.684450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 311/624] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:42:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:05.686251 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240aa76-ca45-4d63-bbdc-5df97b7f9867 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.686763 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240aa76-ca45-4d63-bbdc-5df97b7f9867 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.687277 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240aa76-ca45-4d63-bbdc-5df97b7f9867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.704548 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-2240aa76-ca45-4d63-bbdc-5df97b7f9867 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:05.707801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 314/625] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:42:05.708263 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-2240aa76-ca45-4d63-bbdc-5df97b7f9867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:05.708713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0ae0-3810-4c3d-b4fc-55aeeed29e2f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:05.802738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 312/626] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:05.802738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240aa76-ca45-4d63-bbdc-5df97b7f9867 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:05.847645 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0ae0-3810-4c3d-b4fc-55aeeed29e2f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:05.888280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:05.889168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.889468 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.889755 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.905509 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:05.907516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:05.923768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240aa76-ca45-4d63-bbdc-5df97b7f9867 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:05.934602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:05.935711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:05.936757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:05.936757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:05.936757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:05.936757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:05.937384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 315/627] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:05.938195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:05.963926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c1088-8253-4c6b-9871-18d5907f21ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:05.964744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c1088-8253-4c6b-9871-18d5907f21ff None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.965057 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c1088-8253-4c6b-9871-18d5907f21ff None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.965356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c1088-8253-4c6b-9871-18d5907f21ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:05.975785 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c1088-8253-4c6b-9871-18d5907f21ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:05.977272 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c1088-8253-4c6b-9871-18d5907f21ff None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-346c1088-8253-4c6b-9871-18d5907f21ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:05.995510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:05.996570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:05.998226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:05.998226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:05.998226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:05.998226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:05.998226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 313/628] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:05.998226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f489998-87b2-437f-9d61-df6b5766f255 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:05.999083 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f489998-87b2-437f-9d61-df6b5766f255 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:05.999083 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f489998-87b2-437f-9d61-df6b5766f255 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.001125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 314/629] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:06.002181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f489998-87b2-437f-9d61-df6b5766f255 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.047386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087df28-f380-44c7-84ad-0aa3ff5ca6a1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.082418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbee345-d15a-4bed-aab5-382255371bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.083281 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbee345-d15a-4bed-aab5-382255371bb0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.083574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbee345-d15a-4bed-aab5-382255371bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.083906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbee345-d15a-4bed-aab5-382255371bb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.094176 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbee345-d15a-4bed-aab5-382255371bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-ccbee345-d15a-4bed-aab5-382255371bb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:06.102356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:06.103612 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 316/630] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:06.104994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbee345-d15a-4bed-aab5-382255371bb0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.115501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f489998-87b2-437f-9d61-df6b5766f255 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.148510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce5026-f41e-4cc3-95c8-445a637569b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.149214 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce5026-f41e-4cc3-95c8-445a637569b9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.149509 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce5026-f41e-4cc3-95c8-445a637569b9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.149796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce5026-f41e-4cc3-95c8-445a637569b9 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.150527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce5026-f41e-4cc3-95c8-445a637569b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:06.150842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce5026-f41e-4cc3-95c8-445a637569b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:06.152211 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce5026-f41e-4cc3-95c8-445a637569b9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:06.163123 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 315/631] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:06.164385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aef5c-302e-4343-8b46-ba8ea9d9f893 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.164692 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aef5c-302e-4343-8b46-ba8ea9d9f893 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.164983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aef5c-302e-4343-8b46-ba8ea9d9f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.181293 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-1e5aef5c-302e-4343-8b46-ba8ea9d9f893 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:06.182500 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-1e5aef5c-302e-4343-8b46-ba8ea9d9f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:06.201712 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbee345-d15a-4bed-aab5-382255371bb0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.233576 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.234175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.234466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.234749 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.235476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:06.235788 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:06.237177 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:06.246158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 316/632] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:06.247476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b5eb9-8bce-4a33-85c4-3da3fd21173d None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.247773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b5eb9-8bce-4a33-85c4-3da3fd21173d None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.248054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b5eb9-8bce-4a33-85c4-3da3fd21173d None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.249545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 317/633] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:06.250051 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b5eb9-8bce-4a33-85c4-3da3fd21173d None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.257843 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 317/634] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:06.257843 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.350531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b5eb9-8bce-4a33-85c4-3da3fd21173d None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.352153 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075b216-56ac-4765-a706-2b725f1a73e3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.383158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe4ee-b6a5-4161-8476-6706984a6b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.383826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe4ee-b6a5-4161-8476-6706984a6b2e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.384114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe4ee-b6a5-4161-8476-6706984a6b2e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.384396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe4ee-b6a5-4161-8476-6706984a6b2e None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.385175 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe4ee-b6a5-4161-8476-6706984a6b2e None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:06.385294 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.385491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe4ee-b6a5-4161-8476-6706984a6b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:06.386075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.386366 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.386648 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.386878 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe4ee-b6a5-4161-8476-6706984a6b2e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:06.393189 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:06.394764 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:06.394900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 318/635] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:06.398243 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bc19d-3663-414f-a7e7-d207e1276c09 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.398545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bc19d-3663-414f-a7e7-d207e1276c09 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.398969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bc19d-3663-414f-a7e7-d207e1276c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:06.416649 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:06.417845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:06.418858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:06.418858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:06.418858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:06.418858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:06.418858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:06.418858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:06.418858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:06.418858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:06.419353 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-d93bc19d-3663-414f-a7e7-d207e1276c09 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:06.419449 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 318/636] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:06.419527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-d93bc19d-3663-414f-a7e7-d207e1276c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:06.419610 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.482276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 319/637] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:06.483113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bc19d-3663-414f-a7e7-d207e1276c09 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.516685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a51b8-6572-41b1-9afa-3b81ad820afb None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.549003 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9251c-380a-450e-8488-5994e638961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.549825 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9251c-380a-450e-8488-5994e638961d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.550119 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9251c-380a-450e-8488-5994e638961d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.550406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9251c-380a-450e-8488-5994e638961d None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.557801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9251c-380a-450e-8488-5994e638961d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:06.559260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9251c-380a-450e-8488-5994e638961d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-80e9251c-380a-450e-8488-5994e638961d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:06.574165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:06.575334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 319/638] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:06.576373 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9251c-380a-450e-8488-5994e638961d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.583441 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bc19d-3663-414f-a7e7-d207e1276c09 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.616845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64e12f-62e0-4194-9462-88b196b83a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.617689 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64e12f-62e0-4194-9462-88b196b83a36 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.617981 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64e12f-62e0-4194-9462-88b196b83a36 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.618266 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64e12f-62e0-4194-9462-88b196b83a36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.628152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64e12f-62e0-4194-9462-88b196b83a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-2d64e12f-62e0-4194-9462-88b196b83a36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:06.636256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:06.637499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:06.638489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 320/639] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:06.639275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64e12f-62e0-4194-9462-88b196b83a36 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.674855 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9251c-380a-450e-8488-5994e638961d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.707288 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.708171 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.708530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.708815 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.718631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:06.720065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:06.733341 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64e12f-62e0-4194-9462-88b196b83a36 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:06.737782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:06.739335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:06.740326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:06.740326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:06.740326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:06.740326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:06.740326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 320/640] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:06.740654 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.770307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.770907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.771196 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.771480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.772275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:06.772592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:06.774056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:06.786754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 321/641] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:06.787631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.840466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0f071-b2ad-45ed-b42f-2f3eaba9cca8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.872863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77528dd7-5ee0-44be-bebc-20aa6bb51719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.873678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77528dd7-5ee0-44be-bebc-20aa6bb51719 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.873970 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77528dd7-5ee0-44be-bebc-20aa6bb51719 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.874254 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77528dd7-5ee0-44be-bebc-20aa6bb51719 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.881536 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7f51-8bde-4324-a6cc-ad6d4c708430 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:06.884052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77528dd7-5ee0-44be-bebc-20aa6bb51719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-77528dd7-5ee0-44be-bebc-20aa6bb51719 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:06.891738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:06.893236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 321/642] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:06.894187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77528dd7-5ee0-44be-bebc-20aa6bb51719 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.919650 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:06.920328 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:06.920615 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:06.920895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:06.921737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:06.922055 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:06.923488 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:06.934698 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 322/643] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:06.935502 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:06.994215 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77528dd7-5ee0-44be-bebc-20aa6bb51719 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.026109 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.026716 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.027006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.027292 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.028044 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] RBAC: Adding query filter params (name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:07.028360 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.029059 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e288cf-1b78-48b2-adb9-d3ac202db931 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.029760 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.038132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 322/644] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:07.038900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.061506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.062172 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.062461 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.062746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.063476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:07.063807 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.065190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.076418 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 323/645] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:07.077240 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.136958 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300583-5131-445a-b4da-dcf1441a6470 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.168721 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1bde3-100f-49ae-b236-7c4d16e04b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.169640 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1bde3-100f-49ae-b236-7c4d16e04b64 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.169928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1bde3-100f-49ae-b236-7c4d16e04b64 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.170212 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1bde3-100f-49ae-b236-7c4d16e04b64 None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/48c22f093f45490e996d673b027bf391/roles/d28cdc73a0fc42f6b6e0c45768d418ae` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.172815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24d2db-dd85-4e92-8f2f-65bce4da04e2 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.183889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1bde3-100f-49ae-b236-7c4d16e04b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=48c22f093f45490e996d673b027bf391, role_id=d28cdc73a0fc42f6b6e0c45768d418ae)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.185367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1bde3-100f-49ae-b236-7c4d16e04b64 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.198744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 323/646] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:07 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/48c22f093f45490e996d673b027bf391/roles/d28cdc73a0fc42f6b6e0c45768d418ae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:42:07.199748 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1bde3-100f-49ae-b236-7c4d16e04b64 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.206151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.206904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.207725 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.208018 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.208808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:07.209154 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.210618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.219994 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 324/647] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:07.220891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.312530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1bde3-100f-49ae-b236-7c4d16e04b64 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.328922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d6d02-fdd2-4114-afd8-bad13f5e0b97 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.349868 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.350545 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.350838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.351120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.351983 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=ef8546c698f9455cb81264bcee0d2344) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:07.352308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.353803 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.369073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 324/648] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/role_assignments?user.id=ef8546c698f9455cb81264bcee0d2344&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:07.369073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.376370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.377323 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.377703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.378057 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.395075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.396606 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:07.420898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:07.422028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:07.423066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:07.423066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:07.423066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:07.423066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:07.424113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 325/649] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:07.425315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.497958 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b912b8-f9ee-4cc3-9a7a-05f09376db21 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.537744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.538570 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.538862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.539148 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] PATH_INFO: `/v3/users/glance` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.539261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7b0d9-17de-45de-8bb5-f765c35ad2fe None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.547331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.548807 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:07.565293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:07.567626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 09 15:42:07.569477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:07.571871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 325/650] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:07.572818 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.581716 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.582938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.583470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.583939 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.595715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.597854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:07.620267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:07.621434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:07.623113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:07.623113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:07.623113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:07.623113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:07.623113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 326/651] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:07.623113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.698020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084283-a4b9-45b5-92b8-25f736c1419f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.730694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e7eb2-58e2-48b9-be99-11b1dc8527d1 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.736598 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.737320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.737611 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.737892 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.739186 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:07.739499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.740880 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.752851 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 326/652] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:07.754401 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.766923 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.767667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.767958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.768239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.769188 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:07.770048 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.771565 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.791332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 327/653] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:07.792203 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.862487 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d50d7-b954-448a-bfe8-ae1e8fe4c91a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.893119 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2cedd-a111-4983-9325-4334cdd55d6b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:07.897476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.898151 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.898444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.898735 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.899480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:07.899796 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.901213 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.917164 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 327/654] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:07.918104 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:07.934796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:07.935615 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:07.935905 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:07.936185 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:07.944453 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:07.945998 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:07.961563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:07.964067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 328/655] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:07.966530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:08.020538 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8d951-66d3-4e74-b760-5ea246114fe5 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.056297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa58a-6543-4fc6-b0ae-b0242008b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:08.056965 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa58a-6543-4fc6-b0ae-b0242008b02c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:08.057285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa58a-6543-4fc6-b0ae-b0242008b02c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:08.057572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aa58a-6543-4fc6-b0ae-b0242008b02c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:08.069357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaba94b-1f48-4dbe-a8ae-314796421e55 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.113282 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:08.114140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:08.114590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:08.114881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:08.124514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:08.128239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:08.132627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3aa58a-6543-4fc6-b0ae-b0242008b02c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:08.134810 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3aa58a-6543-4fc6-b0ae-b0242008b02c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:08.145919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:08.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 09 15:42:08.148063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:08.151789 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 329/656] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:08.152668 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:08.257875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c330fe-826d-4e1d-8397-a51727e1b552 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.264122 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 328/657] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:42:08.265678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa58a-6543-4fc6-b0ae-b0242008b02c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:08.295475 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:08.296226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:08.296516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:08.296799 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:08.297742 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:08.298060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:08.299504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:08.318440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 330/658] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 9 15:42:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:08.319387 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:08.390932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aa58a-6543-4fc6-b0ae-b0242008b02c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.433181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:08.434152 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:08.434444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:08.434731 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:08.435503 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4cea0-5697-4a6a-8419-e3f67b5a347f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.435635 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:08.435905 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:08.437600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:08.459136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 329/659] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 9 15:42:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:08.460078 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:08.478381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:08.479793 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:08.480080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:08.480359 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:08.494220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:08.495845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:08.519591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:08.521988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:08.524223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:08.524223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:08.524223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:08.524223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:08.524773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 331/660] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:08.533093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:08.575272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479caf2d-6fea-4870-bcdf-5fb6346a22d1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.617814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad94ae-ed0c-4bfd-9613-55fadf295d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:08.618475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad94ae-ed0c-4bfd-9613-55fadf295d4f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:08.619214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad94ae-ed0c-4bfd-9613-55fadf295d4f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:08.619506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad94ae-ed0c-4bfd-9613-55fadf295d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:08.632124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf5d6-ebef-4c26-8e0a-66b6586f8c2f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.674961 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2375-1ffe-478a-804b-54ce6085adad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:08.675648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2375-1ffe-478a-804b-54ce6085adad None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:08.675941 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2375-1ffe-478a-804b-54ce6085adad None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:08.676224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2375-1ffe-478a-804b-54ce6085adad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:08.704662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad94ae-ed0c-4bfd-9613-55fadf295d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:08.707557 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad94ae-ed0c-4bfd-9613-55fadf295d4f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:08.751913 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2375-1ffe-478a-804b-54ce6085adad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:08.753421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2375-1ffe-478a-804b-54ce6085adad None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:08.821598 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 330/661] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:42:08.823007 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad94ae-ed0c-4bfd-9613-55fadf295d4f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:08.864285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 332/662] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:42:08.865554 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2375-1ffe-478a-804b-54ce6085adad None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:08.935441 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad94ae-ed0c-4bfd-9613-55fadf295d4f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.968687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2375-1ffe-478a-804b-54ce6085adad None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:08.973594 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:08.974580 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:08.975127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:08.975620 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:08.976561 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:08.977084 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:08.978753 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:08.995718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 331/663] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:08.996903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.006046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.007199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.007199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.007550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.017952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.020101 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:09.038620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:09.039702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:09.041267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:09.041267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:09.041267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:09.041267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:09.044563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 333/664] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:09.045525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.110411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69d418-ec9a-41e8-8f62-a80ff9521c0c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.145678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.146301 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6b367-4055-4f81-ae2b-977c62cc2951 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.147389 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.147874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.148431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.160663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.162422 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.181687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.182576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:09.182900 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:09.183906 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:09.184931 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:09.184931 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:09.184931 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:09.184931 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:09.185208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.185208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.187614 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 332/665] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:09.188696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.190950 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.192634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:09.206703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:09.208281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:09.209375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:09.209375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:09.209375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:09.209375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:09.209375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:09.209375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:09.209375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:09.209375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:09.213238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 334/666] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:09.214294 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.301523 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc320e09-f949-451d-bdf9-19e73381251a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.314901 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd845cf-2982-4a9b-b57a-90abfe9209f2 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.337483 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.339010 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.339775 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.340496 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.341704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:09.342276 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.343968 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.351845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.352649 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.353010 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.353408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.354588 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:09.354588 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.356122 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.361264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 333/667] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:09.362623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.375375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 335/668] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:09.376800 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.475791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd1423-480e-4ad8-b076-ffe193c95479 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.477014 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef7b5f-0403-4211-90fb-33a1d1221f5f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.511758 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.512127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.512799 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.513169 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.513521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.513936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.514411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.514894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.515892 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:09.516397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.518678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.522250 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.523840 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.533553 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 334/669] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:09.534925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:09.542938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:09.543969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:09.544983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:09.544983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:09.544983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:09.544983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:09.544983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:09.544983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:09.544983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:09.544983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:09.547596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 336/670] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:09.548855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.637994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad35d63-5260-45ad-b7de-e952165024a8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.654354 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae77b96-ef17-4b5e-9247-3551f9799dda None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.673782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.674590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.674887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.675169 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.682005 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.683448 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.687893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dafbab-f7bb-409f-80d0-84d7f3332b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.688556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafbab-f7bb-409f-80d0-84d7f3332b4c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.688842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafbab-f7bb-409f-80d0-84d7f3332b4c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.689147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafbab-f7bb-409f-80d0-84d7f3332b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.689983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dafbab-f7bb-409f-80d0-84d7f3332b4c None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=4ff3fa04f3f749debec784ed5d2818a5) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:09.690300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dafbab-f7bb-409f-80d0-84d7f3332b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.692284 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dafbab-f7bb-409f-80d0-84d7f3332b4c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.705136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 337/671] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/role_assignments?user.id=4ff3fa04f3f749debec784ed5d2818a5&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:09.705384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:09.706359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:09.707982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:09.707982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:09.707982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:09.707982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:09.707982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:09.707982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:09.707982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:09.707982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:09.708487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb76b42-88f7-4b03-9f2d-60b28dc82073 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.708576 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 335/672] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:09.708654 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb76b42-88f7-4b03-9f2d-60b28dc82073 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.708654 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb76b42-88f7-4b03-9f2d-60b28dc82073 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.708809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.708893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 338/673] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:09.708893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b02704-cab9-43c6-86de-533c692d10bf None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.708893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b02704-cab9-43c6-86de-533c692d10bf None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.708893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b02704-cab9-43c6-86de-533c692d10bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.737918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-e6b02704-cab9-43c6-86de-533c692d10bf None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:09.739347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-e6b02704-cab9-43c6-86de-533c692d10bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:09.836805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 339/674] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:09.837591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc70e0c-24fc-45f9-96b0-179d0fd5ac39 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.837709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b02704-cab9-43c6-86de-533c692d10bf None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.882944 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09afb95-6706-47a8-88fb-5af8ea3f99e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.883622 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09afb95-6706-47a8-88fb-5af8ea3f99e2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.883910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09afb95-6706-47a8-88fb-5af8ea3f99e2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.884193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09afb95-6706-47a8-88fb-5af8ea3f99e2 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.885008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09afb95-6706-47a8-88fb-5af8ea3f99e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:09.885357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09afb95-6706-47a8-88fb-5af8ea3f99e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:09.886831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09afb95-6706-47a8-88fb-5af8ea3f99e2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:09.901372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 336/675] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:09.903216 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8421e1a-90d8-41bb-9ba2-b9e5266334c2 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.906277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8421e1a-90d8-41bb-9ba2-b9e5266334c2 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.906277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8421e1a-90d8-41bb-9ba2-b9e5266334c2 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.906277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 337/676] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:09.906277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8421e1a-90d8-41bb-9ba2-b9e5266334c2 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:09.958730 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b02704-cab9-43c6-86de-533c692d10bf None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:09.996491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:09.997190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:09.997489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:09.997775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:09.998582 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:09.998928 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.000381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.012269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 340/677] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:10.013220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.031263 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8421e1a-90d8-41bb-9ba2-b9e5266334c2 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.065967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f42da-b8cd-4be0-9c65-e36775eddd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.066803 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f42da-b8cd-4be0-9c65-e36775eddd60 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.067093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f42da-b8cd-4be0-9c65-e36775eddd60 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.067376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f42da-b8cd-4be0-9c65-e36775eddd60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.077735 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f42da-b8cd-4be0-9c65-e36775eddd60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-043f42da-b8cd-4be0-9c65-e36775eddd60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:10.086033 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:10.087071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:10.088028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 338/678] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:10.088829 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94002f5c-1673-4869-93c8-b50e5bbb104d None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.088829 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94002f5c-1673-4869-93c8-b50e5bbb104d None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.088998 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94002f5c-1673-4869-93c8-b50e5bbb104d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.106097 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-94002f5c-1673-4869-93c8-b50e5bbb104d None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:10.107285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-94002f5c-1673-4869-93c8-b50e5bbb104d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:10.126721 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef4f3b-49f8-4fca-9fd2-04e72cdbf509 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.160906 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.161625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.161895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.162179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.163032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=8c4f76337b10446999b1e9eef4495ef7) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:10.163353 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.164793 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.181075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 341/679] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/role_assignments?user.id=8c4f76337b10446999b1e9eef4495ef7&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:10.181075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.183962 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 339/680] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:10.184961 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94002f5c-1673-4869-93c8-b50e5bbb104d None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.313119 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8f82-e633-4ca4-9e08-9118711cec78 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.320691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94002f5c-1673-4869-93c8-b50e5bbb104d None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.369341 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.370037 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.370329 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.370618 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.371392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] RBAC: Adding query filter params (name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:10.371713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.373187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.374043 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.375207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.376082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.376595 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.377713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=88be715120ee4267a4a206b71fa21224) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:10.378270 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.381089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.387988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 342/682] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/role_assignments?user.id=88be715120ee4267a4a206b71fa21224&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:10.388326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 340/682] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:10.388861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.390891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.517199 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac75b6-0386-4a01-926e-9061b0088c80 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.532762 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed8835-6c3a-4b5b-ad8a-8cdfc1fe19b6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.562423 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.564363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.565163 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.565759 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.579989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80018c46-318c-4e87-93cc-2e346eb8d76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.580241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.581402 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018c46-318c-4e87-93cc-2e346eb8d76e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.581622 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.582164 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018c46-318c-4e87-93cc-2e346eb8d76e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.582665 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80018c46-318c-4e87-93cc-2e346eb8d76e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.594901 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80018c46-318c-4e87-93cc-2e346eb8d76e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:10.605110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:10.606175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:10.607181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:10.607181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:10.607181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:10.607181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:10.607181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 341/683] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-80018c46-318c-4e87-93cc-2e346eb8d76e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:10.609092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:10.610221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:10.611139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 343/684] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:10.611923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.616337 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80018c46-318c-4e87-93cc-2e346eb8d76e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.733872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1b51d-82b4-4d4c-b71d-39d14392fb7b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.740870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80018c46-318c-4e87-93cc-2e346eb8d76e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.770159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.770724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.771020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.771306 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.772048 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:10.772415 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.773862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.777420 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.778345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.778825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.779294 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.780222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:10.780699 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.782263 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.788166 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 342/685] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:10.791455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 344/686] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:10.793700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.799063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.904006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1a90e-2069-4cdb-a6cd-603cf894e9f6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.911677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f1409-41e3-4f46-9301-3ec119d34abf None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:10.939780 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.940597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.940891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.941256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.952336 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.953570 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:10.953883 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.954371 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:10.954656 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:10.954933 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] PATH_INFO: `/v3/users/swift` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:10.962554 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:10.963995 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:10.971744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:10.973632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:10.974629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:10.974629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:10.974629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:10.974629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:10.978442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 345/687] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:10.979634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:10.981217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:10.983424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:10.985879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 343/688] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:10.997063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:11.094010 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9637b-8569-437e-a924-7bcb6a820468 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:11.132083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:11.132763 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:11.133064 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:11.133358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:11.134203 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:11.134516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:11.135943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:11.157498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 344/689] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:11.158742 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:11.166481 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7833d19-5eae-46ee-83f3-b398762f93ce None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:11.255589 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:11.256333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:11.256630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:11.256913 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:11.257697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:11.258015 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:11.259399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:11.286157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8c9e-bbf4-4141-a80f-694437d44b82 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:11.295330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 346/690] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:11.302401 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:11.341557 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceebc06-dca2-4af3-b169-aef9b81b677a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:11.342222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceebc06-dca2-4af3-b169-aef9b81b677a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:11.342515 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceebc06-dca2-4af3-b169-aef9b81b677a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:11.342798 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceebc06-dca2-4af3-b169-aef9b81b677a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:11.442259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb7b45-5a28-45d6-b463-4f2253ab9a10 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:11.483980 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:11.484843 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:11.485162 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:11.485449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:11.507347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceebc06-dca2-4af3-b169-aef9b81b677a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:11.508883 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceebc06-dca2-4af3-b169-aef9b81b677a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:11.509860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:11.511333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:11.539447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:11.540550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:11.541672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 347/691] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:42:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:11.545991 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:11.629347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 345/692] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:42:11.635676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceebc06-dca2-4af3-b169-aef9b81b677a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:11.678262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b76b8-f7c2-4a0d-909a-330eb1ac8d49 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:11.729513 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:11.730553 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:11.731089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:11.731580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:11.732646 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:11.734345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:11.738502 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:11.776883 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 348/693] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:42:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:11.781991 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:11.801019 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceebc06-dca2-4af3-b169-aef9b81b677a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:11.840681 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:11.841941 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:11.843529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:11.844027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:11.863806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:11.866983 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:11.893465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:11.895141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:11.896165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:11.896165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:11.896165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:11.896165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:11.898482 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 346/694] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:11.907656 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:11.970623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe4d60-d1ab-44c6-83de-1de5aa86aaaa None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:12.017996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa47a1-49fd-494b-bb7a-e59f49d683aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:12.018601 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa47a1-49fd-494b-bb7a-e59f49d683aa None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.018896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa47a1-49fd-494b-bb7a-e59f49d683aa None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.019209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa47a1-49fd-494b-bb7a-e59f49d683aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.063847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16ce7d-efe0-4fef-9f54-7bc39703b623 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:12.104839 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:12.109060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.109060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.109060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.109060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:12.109060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:12.110935 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:12.126220 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 347/695] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:12.132670 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:12.171180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa47a1-49fd-494b-bb7a-e59f49d683aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:12.172658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa47a1-49fd-494b-bb7a-e59f49d683aa None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:12.307401 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950577-9a09-4faf-9f8a-89c1eea8566f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:12.324860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 349/696] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 09 15:42:12.330067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa47a1-49fd-494b-bb7a-e59f49d683aa None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:12.360750 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71780100-a1e8-4107-92fd-f159ff46755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:12.361436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71780100-a1e8-4107-92fd-f159ff46755d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.361731 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71780100-a1e8-4107-92fd-f159ff46755d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.362027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71780100-a1e8-4107-92fd-f159ff46755d None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.378311 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71780100-a1e8-4107-92fd-f159ff46755d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:12.380367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71780100-a1e8-4107-92fd-f159ff46755d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-71780100-a1e8-4107-92fd-f159ff46755d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:12.409062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:12.410876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 348/697] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e5b3-7c2a-45e4-8bcf-5cf13beec34f None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.413022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e5b3-7c2a-45e4-8bcf-5cf13beec34f None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.413895 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e5b3-7c2a-45e4-8bcf-5cf13beec34f None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.413895 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 349/698] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:12.414971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661622d1-de77-42a7-bed8-6312ecfe76ad None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.414971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661622d1-de77-42a7-bed8-6312ecfe76ad None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.416710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661622d1-de77-42a7-bed8-6312ecfe76ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.434529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-661622d1-de77-42a7-bed8-6312ecfe76ad None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:12.435943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-661622d1-de77-42a7-bed8-6312ecfe76ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:12.442189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa47a1-49fd-494b-bb7a-e59f49d683aa None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:12.477380 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:12.478184 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.478470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.478749 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.490209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:12.491741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:12.509907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:12.512054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:12.513940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:12.513940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:12.513940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:12.513940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:12.513940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 350/700] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:12.514561 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 350/700] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:42:12.514707 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:12.519067 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661622d1-de77-42a7-bed8-6312ecfe76ad None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:12.616037 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83a558-452e-465c-a587-8a2d492d5464 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:12.635268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661622d1-de77-42a7-bed8-6312ecfe76ad None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:12.650452 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:12.651290 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.651639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.651987 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.652904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:12.653330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:12.654825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:12.667629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 351/701] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:12.668494 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:12.677489 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:12.678184 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.678590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.678932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.679821 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:12.680197 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:12.682146 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:12.697073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 351/702] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:12.698007 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:12.801690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9128b-2cea-4169-a518-0a33b0def723 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:12.825232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2674e8-978c-4e1d-9baa-78585f0a47d8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:12.844743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:12.845918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.846274 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.846568 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.861704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:12.863366 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:12.869050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:12.869570 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:12.869860 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:12.870582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:12.871444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=48c22f093f45490e996d673b027bf391) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:12.872167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:12.874002 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:12.881442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:12.882747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:12.884719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:12.884719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:12.884719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:12.884719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:12.884719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 352/704] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:12.885363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 352/704] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:12 2022] GET /identity/v3/role_assignments?user.id=48c22f093f45490e996d673b027bf391&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:12.893071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:12.895637 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.043170 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c24ae6-27c4-48c4-9d1c-3ffa14f520ae None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.057280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9db17-af3d-4a77-b7fd-a413f0a67ce6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.082967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6fb2c-2875-499a-ad50-4a8c6d2996c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.083870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6fb2c-2875-499a-ad50-4a8c6d2996c7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.084263 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6fb2c-2875-499a-ad50-4a8c6d2996c7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.084455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6fb2c-2875-499a-ad50-4a8c6d2996c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.092376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6fb2c-2875-499a-ad50-4a8c6d2996c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.093904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6fb2c-2875-499a-ad50-4a8c6d2996c7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.107530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2f7ec-732d-4adb-9060-d10581093865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.108249 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2f7ec-732d-4adb-9060-d10581093865 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.108541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2f7ec-732d-4adb-9060-d10581093865 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.108825 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2f7ec-732d-4adb-9060-d10581093865 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.109618 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2f7ec-732d-4adb-9060-d10581093865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:13.109928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2f7ec-732d-4adb-9060-d10581093865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-66f6fb2c-2875-499a-ad50-4a8c6d2996c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:13.110988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:13.111951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:13.113149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:13.113149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:13.113149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:13.113149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:13.113149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:13.113149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:13.113149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:13.113149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:13.113639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2f7ec-732d-4adb-9060-d10581093865 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.113718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 353/705] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:13.114000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d2e11-4edb-4f53-8414-1bfdf3560698 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.114295 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d2e11-4edb-4f53-8414-1bfdf3560698 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.114580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d2e11-4edb-4f53-8414-1bfdf3560698 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.120642 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 354/706] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:13.120918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d2e11-4edb-4f53-8414-1bfdf3560698 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.126032 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 353/707] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:13.127597 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45dfb0-ffb7-453a-ab0b-a7d184dfaa5b None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.127889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45dfb0-ffb7-453a-ab0b-a7d184dfaa5b None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.128168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45dfb0-ffb7-453a-ab0b-a7d184dfaa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.153327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-0c45dfb0-ffb7-453a-ab0b-a7d184dfaa5b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:13.154097 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-0c45dfb0-ffb7-453a-ab0b-a7d184dfaa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:13.234947 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 354/708] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:13.235805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45dfb0-ffb7-453a-ab0b-a7d184dfaa5b None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.262704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d2e11-4edb-4f53-8414-1bfdf3560698 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.298639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.299314 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.299601 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.299885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.300718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:13.301031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.302503 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.314020 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 355/709] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:13.315089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.351205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45dfb0-ffb7-453a-ab0b-a7d184dfaa5b None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.385911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3610e7-b5f2-4fe7-b5fa-0bb7ceddd1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.386937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3610e7-b5f2-4fe7-b5fa-0bb7ceddd1ba None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.387461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3610e7-b5f2-4fe7-b5fa-0bb7ceddd1ba None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.387956 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3610e7-b5f2-4fe7-b5fa-0bb7ceddd1ba None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.388873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3610e7-b5f2-4fe7-b5fa-0bb7ceddd1ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.390530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3610e7-b5f2-4fe7-b5fa-0bb7ceddd1ba None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.411365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 355/710] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:42:13 2022] POST /identity/v3/users => generated 254 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:13.413019 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcfb8f-3fbf-43c8-a72a-46d701ac8d33 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.413536 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcfb8f-3fbf-43c8-a72a-46d701ac8d33 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.413990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcfb8f-3fbf-43c8-a72a-46d701ac8d33 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.416261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 356/711] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:13.417388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcfb8f-3fbf-43c8-a72a-46d701ac8d33 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.429375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd42c6-2aac-4419-bc72-015531e43fce None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.473496 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011476a-0e93-40d3-931b-519254ef40f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.475864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011476a-0e93-40d3-931b-519254ef40f5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.476397 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011476a-0e93-40d3-931b-519254ef40f5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.476910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011476a-0e93-40d3-931b-519254ef40f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.477859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011476a-0e93-40d3-931b-519254ef40f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.479606 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011476a-0e93-40d3-931b-519254ef40f5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-7011476a-0e93-40d3-931b-519254ef40f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:13.500592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:13.501867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:13.502881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:13.507039 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 356/712] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:42:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:13.508641 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5144-0c3f-41dd-be4c-1b1d6389d7a3 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.509124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5144-0c3f-41dd-be4c-1b1d6389d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.509769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5144-0c3f-41dd-be4c-1b1d6389d7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.536345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-732c5144-0c3f-41dd-be4c-1b1d6389d7a3 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:13.538132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-732c5144-0c3f-41dd-be4c-1b1d6389d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:13.559644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcfb8f-3fbf-43c8-a72a-46d701ac8d33 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.597524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.598476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.598957 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.599403 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.600434 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=ef8546c698f9455cb81264bcee0d2344) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:13.600919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.602563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.613533 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 357/713] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:13 2022] GET /identity/v3/role_assignments?user.id=ef8546c698f9455cb81264bcee0d2344&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=ebff922955b74c919d58613875326151 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:13.614730 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.628545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 357/714] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:13.653843 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c5144-0c3f-41dd-be4c-1b1d6389d7a3 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.734631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e353-5808-4769-8342-d311623d2fa3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.764136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c5144-0c3f-41dd-be4c-1b1d6389d7a3 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.772516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.773242 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.773530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.773813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.774544 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:13.774858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.776325 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.781639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 358/715] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:42:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:13.788925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.808284 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.809142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.809443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.809723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] PATH_INFO: `/v3/services/network` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.810454 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.811918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:13.827421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:13.828626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:13.829617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 358/716] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:13 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:13.836079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:13.902336 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb5de7-6fe0-46a0-a283-14341e55c883 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.937706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.938383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.938675 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.938958 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.939682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:13.939996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.941440 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.945504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41c713-3a26-44bf-80ed-cfb1f1d0261b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:13.947514 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 359/717] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:42:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:13.987501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d805048-40b7-490f-ae45-ac283e6f0e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:13.988766 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d805048-40b7-490f-ae45-ac283e6f0e09 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:13.989097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d805048-40b7-490f-ae45-ac283e6f0e09 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:13.989363 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d805048-40b7-490f-ae45-ac283e6f0e09 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:13.990109 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d805048-40b7-490f-ae45-ac283e6f0e09 None admin] RBAC: Adding query filter params (name=network) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:13.990422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d805048-40b7-490f-ae45-ac283e6f0e09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:13.991853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d805048-40b7-490f-ae45-ac283e6f0e09 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:13.997453 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 359/718] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:14.002587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:14.133114 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ae6ba-948a-44bb-bdda-c01f5516c2f7 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:14.170072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b7a76-2d08-4834-b3ac-585e905be2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:14.170673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7a76-2d08-4834-b3ac-585e905be2cd None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:14.170965 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7a76-2d08-4834-b3ac-585e905be2cd None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:14.171252 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b7a76-2d08-4834-b3ac-585e905be2cd None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:14.171993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b7a76-2d08-4834-b3ac-585e905be2cd None admin] RBAC: Adding query filter params (type=network) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:14.172308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b7a76-2d08-4834-b3ac-585e905be2cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:14.173796 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b7a76-2d08-4834-b3ac-585e905be2cd None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:14.179932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 360/719] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:15.562623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69452-920e-4618-a02b-fc12170fa750 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:15.563153 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69452-920e-4618-a02b-fc12170fa750 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:15.563237 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69452-920e-4618-a02b-fc12170fa750 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:15.564814 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 360/720] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:15.570550 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98caed-f062-468f-a3ab-ca7b40788dde None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:15.570981 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98caed-f062-468f-a3ab-ca7b40788dde None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:15.571167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98caed-f062-468f-a3ab-ca7b40788dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:15.593583 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-3f98caed-f062-468f-a3ab-ca7b40788dde None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:15.594795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-3f98caed-f062-468f-a3ab-ca7b40788dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:15.692304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 361/721] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 09 15:42:15.740911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69452-920e-4618-a02b-fc12170fa750 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:15.856514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69452-920e-4618-a02b-fc12170fa750 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:15.889757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a0f7d-897e-4a90-a002-d6330a4cb592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:15.890557 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a0f7d-897e-4a90-a002-d6330a4cb592 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:15.890842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a0f7d-897e-4a90-a002-d6330a4cb592 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:15.891238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a0f7d-897e-4a90-a002-d6330a4cb592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:15.901999 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a0f7d-897e-4a90-a002-d6330a4cb592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-2f8a0f7d-897e-4a90-a002-d6330a4cb592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:15.912900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:15.914056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:15.915023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 361/722] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:15.920849 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98caed-f062-468f-a3ab-ca7b40788dde None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:16.071355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98caed-f062-468f-a3ab-ca7b40788dde None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:16.114993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2510229-5ee6-4c0a-8ca1-f87dd8c4a82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:16.115585 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2510229-5ee6-4c0a-8ca1-f87dd8c4a82f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.115875 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2510229-5ee6-4c0a-8ca1-f87dd8c4a82f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.116153 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2510229-5ee6-4c0a-8ca1-f87dd8c4a82f None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.116916 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2510229-5ee6-4c0a-8ca1-f87dd8c4a82f None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:16.117256 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2510229-5ee6-4c0a-8ca1-f87dd8c4a82f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:16.118626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2510229-5ee6-4c0a-8ca1-f87dd8c4a82f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:16.131730 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 362/723] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:16.137430 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a0f7d-897e-4a90-a002-d6330a4cb592 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:16.238892 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a0f7d-897e-4a90-a002-d6330a4cb592 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:16.273163 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:16.273978 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.274261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.274542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.278985 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38d5a0-54ea-448e-8a8f-db10c1676a12 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.279277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38d5a0-54ea-448e-8a8f-db10c1676a12 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.279556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38d5a0-54ea-448e-8a8f-db10c1676a12 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.280919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 363/724] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:16.286292 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51399-c659-4752-be3f-6d514bc16dbe None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.286620 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51399-c659-4752-be3f-6d514bc16dbe None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.286942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc51399-c659-4752-be3f-6d514bc16dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.289886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:16.291346 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:16.308563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:16.310852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:16.312685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:16.312685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:16.312685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:16.312685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:16.312685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 362/725] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:16.317083 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:16.317386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-3dc51399-c659-4752-be3f-6d514bc16dbe None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:16.318572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-3dc51399-c659-4752-be3f-6d514bc16dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:16.441455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 364/726] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:16.484798 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51399-c659-4752-be3f-6d514bc16dbe None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:16.512250 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd29e-cea6-493e-b89e-f86fe9b3079b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:16.577841 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:16.578895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.579039 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.579279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.580021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:16.580342 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:16.581799 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:16.603010 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 363/727] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:16.607963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:16.626457 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc51399-c659-4752-be3f-6d514bc16dbe None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:16.662893 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f4590-f11a-4723-b610-f90d0f0797d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:16.664111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4590-f11a-4723-b610-f90d0f0797d9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.664618 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4590-f11a-4723-b610-f90d0f0797d9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.665261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4590-f11a-4723-b610-f90d0f0797d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.676891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f4590-f11a-4723-b610-f90d0f0797d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-af9f4590-f11a-4723-b610-f90d0f0797d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:16.684989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:16.686285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:16.687455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:16.689901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 365/728] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:16.696480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a581d-2d35-4eff-b427-bc1f89b8ab12 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.697054 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a581d-2d35-4eff-b427-bc1f89b8ab12 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.697508 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a581d-2d35-4eff-b427-bc1f89b8ab12 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.699262 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 366/729] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:16.700149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a581d-2d35-4eff-b427-bc1f89b8ab12 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:16.715968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a4aaa-97c9-421c-8de2-74712f3ee136 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:16.751360 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a454f96-16f1-4881-b4d6-bd7b0e08e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:16.752440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a454f96-16f1-4881-b4d6-bd7b0e08e997 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.752911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a454f96-16f1-4881-b4d6-bd7b0e08e997 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.753428 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a454f96-16f1-4881-b4d6-bd7b0e08e997 None admin] PATH_INFO: `/v3/users/swift` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.761668 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a454f96-16f1-4881-b4d6-bd7b0e08e997 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:16.763366 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a454f96-16f1-4881-b4d6-bd7b0e08e997 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5a454f96-16f1-4881-b4d6-bd7b0e08e997 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:16.779262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:16.780622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:16.781954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:16.784305 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 364/730] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:16.785957 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5af55d-21a9-4be6-80fe-1b36bb89fb61 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.786411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5af55d-21a9-4be6-80fe-1b36bb89fb61 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.786874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5af55d-21a9-4be6-80fe-1b36bb89fb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.810983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-2b5af55d-21a9-4be6-80fe-1b36bb89fb61 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:16.812822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-2b5af55d-21a9-4be6-80fe-1b36bb89fb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:16.819405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a581d-2d35-4eff-b427-bc1f89b8ab12 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:16.865108 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:16.866871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:16.867347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:16.867794 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:16.869343 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:16.869842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:16.872154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:16.889562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 367/731] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:16.889562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:16.921500 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 365/732] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:16.922341 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5af55d-21a9-4be6-80fe-1b36bb89fb61 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.005377 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67722698-8cd3-4fa6-bbd9-4d31c414c3a7 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.039123 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.039802 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.040033 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5af55d-21a9-4be6-80fe-1b36bb89fb61 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.040151 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.040352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.041231 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:17.041549 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:17.042976 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:17.062755 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 368/733] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:42:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:17.064023 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.078027 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25dff3-f3ed-497e-949f-b9948e3f7b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.078840 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25dff3-f3ed-497e-949f-b9948e3f7b1c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.079133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25dff3-f3ed-497e-949f-b9948e3f7b1c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.079419 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25dff3-f3ed-497e-949f-b9948e3f7b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.089340 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25dff3-f3ed-497e-949f-b9948e3f7b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:17.090797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25dff3-f3ed-497e-949f-b9948e3f7b1c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-af25dff3-f3ed-497e-949f-b9948e3f7b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:17.108399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:17.109590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:17.110685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:17.110685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:17.110685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:17.110685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:17.110685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 366/734] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:17.111270 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ebb97-840d-44af-8c5f-946c8405f120 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.111395 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ebb97-840d-44af-8c5f-946c8405f120 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.111683 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ebb97-840d-44af-8c5f-946c8405f120 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.113667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 367/735] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:17.114626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f48306-1212-4194-a5ef-73a39ed3868e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.114920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f48306-1212-4194-a5ef-73a39ed3868e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.115265 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f48306-1212-4194-a5ef-73a39ed3868e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.120356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 368/736] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:17.120356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f48306-1212-4194-a5ef-73a39ed3868e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.163804 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d257c-d0db-4469-84bc-7ae51653a76f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.195811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a2285-72b6-4f7e-9ca8-f6354df26a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.196595 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a2285-72b6-4f7e-9ca8-f6354df26a3e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.196878 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a2285-72b6-4f7e-9ca8-f6354df26a3e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.197190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a2285-72b6-4f7e-9ca8-f6354df26a3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.206889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a2285-72b6-4f7e-9ca8-f6354df26a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:17.213181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f48306-1212-4194-a5ef-73a39ed3868e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-693a2285-72b6-4f7e-9ca8-f6354df26a3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:17.214537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:17.215719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 369/737] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:17.216831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a2285-72b6-4f7e-9ca8-f6354df26a3e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.249796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9e97c-8717-4689-9397-4db63f973f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.250461 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9e97c-8717-4689-9397-4db63f973f6c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.250748 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9e97c-8717-4689-9397-4db63f973f6c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.251032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9e97c-8717-4689-9397-4db63f973f6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.316099 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9e97c-8717-4689-9397-4db63f973f6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:17.317548 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9e97c-8717-4689-9397-4db63f973f6c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:17.327309 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a2285-72b6-4f7e-9ca8-f6354df26a3e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.361527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37023a8c-d163-4674-8edd-48b905ae1063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.362197 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37023a8c-d163-4674-8edd-48b905ae1063 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.362488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37023a8c-d163-4674-8edd-48b905ae1063 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.362773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37023a8c-d163-4674-8edd-48b905ae1063 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.363506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37023a8c-d163-4674-8edd-48b905ae1063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:17.363820 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37023a8c-d163-4674-8edd-48b905ae1063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:17.365210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37023a8c-d163-4674-8edd-48b905ae1063 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:17.376600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 370/738] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:17.377895 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2299e1-e534-4fc2-b19d-0c6c4a744f41 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.378195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2299e1-e534-4fc2-b19d-0c6c4a744f41 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.378488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2299e1-e534-4fc2-b19d-0c6c4a744f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.395359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-2e2299e1-e534-4fc2-b19d-0c6c4a744f41 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:17.396546 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-2e2299e1-e534-4fc2-b19d-0c6c4a744f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:17.413015 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 369/739] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 09 15:42:17.414351 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e90ad2-62ad-429c-a46d-71ffdfdb473c None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.414653 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e90ad2-62ad-429c-a46d-71ffdfdb473c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.414943 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e90ad2-62ad-429c-a46d-71ffdfdb473c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.430373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-81e90ad2-62ad-429c-a46d-71ffdfdb473c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:17.431568 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-81e90ad2-62ad-429c-a46d-71ffdfdb473c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:17.500042 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 370/740] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:17.500939 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e90ad2-62ad-429c-a46d-71ffdfdb473c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.582360 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 371/741] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 09 15:42:17.583112 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2299e1-e534-4fc2-b19d-0c6c4a744f41 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.600946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e90ad2-62ad-429c-a46d-71ffdfdb473c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.637627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.638308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.638600 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.638884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.639674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:17.639989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:17.641394 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:17.649848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 371/742] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:17.650639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.693918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2299e1-e534-4fc2-b19d-0c6c4a744f41 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.726063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.726863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.727150 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.727433 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] PATH_INFO: `/v3/users/nova` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.734867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:17.736297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:17.746297 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f1a60-883b-4494-8cc8-b42f779d2548 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:17.749911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:17.750954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 372/743] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:42:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:17.752046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.779354 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.780155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.780444 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.780752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.791119 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:17.792638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:17.810327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:17.811511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:17.812873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:17.812873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:17.812873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:17.812873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:17.812873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 372/744] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:17.812873 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:17.853798 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbde7f9-6798-41a2-b2ec-aa7b53f1ed6a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:17.885330 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41ef4c-1aed-4542-9484-9384c9103c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:17.886012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ef4c-1aed-4542-9484-9384c9103c6f None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:17.886299 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ef4c-1aed-4542-9484-9384c9103c6f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:17.886578 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41ef4c-1aed-4542-9484-9384c9103c6f None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:17.887317 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41ef4c-1aed-4542-9484-9384c9103c6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:17.888713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41ef4c-1aed-4542-9484-9384c9103c6f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:17.921114 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 373/745] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:42:17 2022] POST /identity/v3/services => generated 240 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:17.921114 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41ef4c-1aed-4542-9484-9384c9103c6f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.038097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0dc35-1079-4c0d-ad21-2ddd2d212d37 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.043126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41ef4c-1aed-4542-9484-9384c9103c6f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.074111 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b93d5-c1b8-4e06-91de-27929da42c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.074805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b93d5-c1b8-4e06-91de-27929da42c5c None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.075195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b93d5-c1b8-4e06-91de-27929da42c5c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.075652 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b93d5-c1b8-4e06-91de-27929da42c5c None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.076244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b93d5-c1b8-4e06-91de-27929da42c5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.077771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b93d5-c1b8-4e06-91de-27929da42c5c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.081025 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.081908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.082207 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.082491 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.094191 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 373/746] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:42:17 2022] POST /identity/v3/services => generated 251 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:18.094627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.095630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b93d5-c1b8-4e06-91de-27929da42c5c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.096153 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:18.125124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:18.126217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:18.127278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:18.127278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:18.127278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:18.127278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:18.127544 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 374/747] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:18.128263 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.236701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b93d5-c1b8-4e06-91de-27929da42c5c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.261398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a401ba1-2dac-46fc-bf8c-aaf40c615c0c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.289063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.290382 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.290872 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.291553 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.293377 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:18.293949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.296556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.305824 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.306688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.307054 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.307446 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.308204 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:18.308524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.310268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.325158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 374/748] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:18.326515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.328126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 375/749] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:18.332700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.445405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215ad09-d4ff-4d09-924f-ed994d251ac2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.453278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b27a3-cb7b-4eb1-bfd8-8ccbd983f180 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.490076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.491323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.491811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.492297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.494106 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.495040 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.495396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.495745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.496883 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:18.497296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.499043 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.505392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.507642 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.517936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 375/750] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:18.519871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:18.535543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:18.536623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:18.537684 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:18.537684 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:18.537684 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:18.537684 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:18.541369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 376/751] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:18.542444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.638208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc621cc-9e6d-4c7d-85dd-9077801615af None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.650484 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c991b6c-4d99-4fc4-b646-43066823785d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.673902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.674774 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.675059 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.676076 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.683205 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.684659 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.688881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.690021 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.690530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.691081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.698952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:18.699532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:18.700468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:18.701515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 376/752] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:18.705769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.706662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:18.725466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:18.726574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:18.727641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:18.731233 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 377/753] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:18.732110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.813662 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc735298-781b-4bff-a5e4-62700ba8a059 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.837384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904af12c-d48c-4294-93ab-e24ebcc539c1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:18.849382 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.850387 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.850933 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.851443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.852399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:18.852928 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.854553 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.867196 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 377/754] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:18.868395 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.876922 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:18.877684 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:18.877974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:18.878333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:18.879152 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:18.879456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:18.880884 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:18.897487 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 378/755] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:18.898350 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:18.972833 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b8db6-6be9-4c98-93c9-c260ca248790 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:19.006235 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:19.007291 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:19.007834 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:19.008328 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:19.009440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:19.009954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:19.011576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:19.022877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263a81f-b38f-49e9-b174-69f31a82943e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:19.030198 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 378/756] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:19.031321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:19.071125 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:19.072197 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:19.072739 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:19.073279 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:19.080958 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:19.083335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:19.098669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:19.099682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:19.100706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:19.100706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:19.100706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:19.100706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:19.100706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:19.100706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:19.100706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:19.100706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:19.103214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 379/757] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:19.104318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:19.154901 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d417394-36ec-4b3f-be04-e16a609c32c9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:19.194225 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:19.195201 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:19.195495 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:19.195774 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:19.196802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=48c22f093f45490e996d673b027bf391) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:19.197528 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:19.199243 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:19.206562 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 379/758] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:19 2022] GET /identity/v3/role_assignments?user.id=48c22f093f45490e996d673b027bf391&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=ebff922955b74c919d58613875326151 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:19.207877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:19.232955 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3cb1a-9077-4987-92cf-e65b5bef3b8c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:19.312920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf742b32-dddc-433b-88e7-1587ffd7ebb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:19.313636 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf742b32-dddc-433b-88e7-1587ffd7ebb3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:19.314064 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf742b32-dddc-433b-88e7-1587ffd7ebb3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:19.314348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf742b32-dddc-433b-88e7-1587ffd7ebb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:19.390403 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf742b32-dddc-433b-88e7-1587ffd7ebb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:19.391885 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf742b32-dddc-433b-88e7-1587ffd7ebb3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:19.430460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8544e0-ee4c-49d7-80dd-da879525dde6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:19.498977 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:19.499574 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:19.499867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:19.500150 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:19.500955 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:19.501300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:19.502731 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:19.530544 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 380/759] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:19.536536 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:19.558756 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 380/760] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:42:19.565659 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf742b32-dddc-433b-88e7-1587ffd7ebb3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:19.664310 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf742b32-dddc-433b-88e7-1587ffd7ebb3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:19.704139 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:19.704940 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:19.705244 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:19.705526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:19.714796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cfff9e-ebfa-4bbc-bcba-1eaa616bf57a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:19.731061 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:19.732607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:19.762087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:19.763131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:19.764157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:19.764157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:19.764157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:19.764157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:19.764157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 381/761] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:19.768240 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803e91-4b71-4cb5-b572-4b03c81adcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:19.768240 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803e91-4b71-4cb5-b572-4b03c81adcac None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:19.768240 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803e91-4b71-4cb5-b572-4b03c81adcac None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:19.768240 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803e91-4b71-4cb5-b572-4b03c81adcac None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/ef8546c698f9455cb81264bcee0d2344/roles/ebff922955b74c919d58613875326151` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:19.769353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:19.769983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a803e91-4b71-4cb5-b572-4b03c81adcac None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=ef8546c698f9455cb81264bcee0d2344, role_id=ebff922955b74c919d58613875326151)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:19.771468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a803e91-4b71-4cb5-b572-4b03c81adcac None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:19.799320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 381/762] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:19 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/ef8546c698f9455cb81264bcee0d2344/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:42:19.879363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75090e0-5136-4cc4-8c0d-eb2f0ee56b59 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:19.927623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:19.928321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:19.928611 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:19.928898 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:19.929681 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:19.930056 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:19.931586 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:19.944678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 382/763] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:19.949861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803e91-4b71-4cb5-b572-4b03c81adcac None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:20.081344 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803e91-4b71-4cb5-b572-4b03c81adcac None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:20.123739 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:20.124562 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:20.124851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:20.125156 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:20.132641 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:20.134297 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:20.150823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:20.153177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:20.155455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 382/764] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:20.161059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:20.311272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4815f-53a8-4e1c-96f7-27ad4323f286 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:20.356270 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883b945-4d6f-411c-bd0b-70abc5c4b9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:20.357351 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883b945-4d6f-411c-bd0b-70abc5c4b9a7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:20.357863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883b945-4d6f-411c-bd0b-70abc5c4b9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:20.358384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883b945-4d6f-411c-bd0b-70abc5c4b9a7 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:20.359437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883b945-4d6f-411c-bd0b-70abc5c4b9a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:20.359979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883b945-4d6f-411c-bd0b-70abc5c4b9a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:20.361675 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883b945-4d6f-411c-bd0b-70abc5c4b9a7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:20.380936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 383/765] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:20.386703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:20.511772 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079fcd7-7648-4d05-adc8-be34fddafe28 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:20.539704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8cd61-b060-4a37-8b19-cdf3b99674ef None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:20.540376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8cd61-b060-4a37-8b19-cdf3b99674ef None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:20.540891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8cd61-b060-4a37-8b19-cdf3b99674ef None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:20.542826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 384/766] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:20.548359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eb45f-6524-4617-b174-c451c02bf037 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:20.548943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eb45f-6524-4617-b174-c451c02bf037 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:20.549474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eb45f-6524-4617-b174-c451c02bf037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:20.558223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:20.559197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:20.559699 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:20.560193 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:20.562058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=6cf6a2d926e9494abb63a9535b547b6e) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:20.562392 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:20.563851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:20.572086 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 383/767] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:20 2022] GET /identity/v3/role_assignments?user.id=6cf6a2d926e9494abb63a9535b547b6e&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:20.576543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-6f4eb45f-6524-4617-b174-c451c02bf037 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:20.578118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-6f4eb45f-6524-4617-b174-c451c02bf037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:20.663978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 385/768] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:20.687939 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:20.773734 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db767f2-e097-4971-a09f-66480db8a54a None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:20.774301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db767f2-e097-4971-a09f-66480db8a54a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:20.774301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db767f2-e097-4971-a09f-66480db8a54a None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:20.776206 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 386/769] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:20.782258 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3bad5-59b2-4c24-8456-b6cc8f8fe176 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:20.782604 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3bad5-59b2-4c24-8456-b6cc8f8fe176 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:20.782832 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3bad5-59b2-4c24-8456-b6cc8f8fe176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:20.805932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd5aa3-4a32-4afb-98bb-fe9e68eecf90 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:20.822099 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-07c3bad5-59b2-4c24-8456-b6cc8f8fe176 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:20.823307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-07c3bad5-59b2-4c24-8456-b6cc8f8fe176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:20.850317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:20.851546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:20.852072 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:20.852569 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] PATH_INFO: `/v3/services/network` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:20.853514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:20.856004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:20.874982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:20.876199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:20.877262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:20.889342 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 384/770] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:20 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:20.893741 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:20.961970 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 387/771] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:21.001430 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3bad5-59b2-4c24-8456-b6cc8f8fe176 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:21.027898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941326e-fa41-4724-b706-110d5c820b7e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:21.077775 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:21.078474 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:21.078769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:21.079058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:21.079790 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] RBAC: Adding query filter params (name=network) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:21.080108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:21.081694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:21.086782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 385/772] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:21.098364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:21.136508 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3bad5-59b2-4c24-8456-b6cc8f8fe176 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:21.182319 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:21.183478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:21.183988 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:21.184481 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:21.185465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:21.187116 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:21.215176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fec6d-2b98-4233-9aae-16b1b542075a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:21.216772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:21.217908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:21.218874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:21.222283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 388/773] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:42:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:21.227819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:21.262348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:21.263042 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:21.263336 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:21.263624 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:21.264415 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] RBAC: Adding query filter params (type=network) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:21.264671 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:21.266097 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:21.283419 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 386/774] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:21.294091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:21.347136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2ef9-8f1e-4569-a059-8e94c15c9570 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:21.389723 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:21.390864 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:21.391408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:21.391939 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:21.392875 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:21.393439 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:21.399255 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:21.407618 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 389/775] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:42:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:21.413093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:21.507715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b87954-4559-4f68-a05b-8f2413257d81 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:21.568571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:21.569280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:21.569572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:21.569854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:21.570667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] RBAC: Adding query filter params (interface=public, service_id=619419e5e708470599fd7ce0bdd57e8f, region_id=RegionOne) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:21.570987 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:21.572364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:21.585518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 387/776] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:42:21 2022] GET /identity/v3/endpoints?service_id=619419e5e708470599fd7ce0bdd57e8f&interface=public®ion_id=RegionOne => generated 194 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:21.587821 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280108-351a-402b-b04c-fd44394885c9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:21.592550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:21.641098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:21.641943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:21.642229 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:21.642680 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:21.643755 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:21.644708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:21.646112 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:21.651861 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 390/777] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:42:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:21.657429 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:21.784856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a9031-fc77-4503-ade1-07b3cb87d9f3 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:21.802041 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb917f1-2e2e-416f-9409-94f8c4418eb2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:21.847877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cc97a-389c-40dc-a986-1bbc51bfc429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:21.848578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cc97a-389c-40dc-a986-1bbc51bfc429 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:21.849954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cc97a-389c-40dc-a986-1bbc51bfc429 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:21.850240 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cc97a-389c-40dc-a986-1bbc51bfc429 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:21.850938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cc97a-389c-40dc-a986-1bbc51bfc429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:21.852316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cc97a-389c-40dc-a986-1bbc51bfc429 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:21.862474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6d5d0-cc29-4028-b98c-a5b1f22e65ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:21.863672 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d5d0-cc29-4028-b98c-a5b1f22e65ce None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:21.863957 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d5d0-cc29-4028-b98c-a5b1f22e65ce None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:21.864298 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6d5d0-cc29-4028-b98c-a5b1f22e65ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:21.865125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 388/778] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:42:21 2022] GET /identity/v3/services => generated 772 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:21.865237 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6d5d0-cc29-4028-b98c-a5b1f22e65ce None admin] RBAC: Adding query filter params (interface=public, service_id=111190850c944036a8341ddda9c2653a, region_id=RegionOne) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:21.865642 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6d5d0-cc29-4028-b98c-a5b1f22e65ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:21.867158 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6d5d0-cc29-4028-b98c-a5b1f22e65ce None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:21.884794 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 391/779] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:42:21 2022] GET /identity/v3/endpoints?service_id=111190850c944036a8341ddda9c2653a&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:21.890689 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cc97a-389c-40dc-a986-1bbc51bfc429 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:22.036331 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cc97a-389c-40dc-a986-1bbc51bfc429 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:22.072379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c129e-7414-41e1-8ade-fb0d02c7d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:22.073399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c129e-7414-41e1-8ade-fb0d02c7d9e2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:22.073838 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c129e-7414-41e1-8ade-fb0d02c7d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:22.074271 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c129e-7414-41e1-8ade-fb0d02c7d9e2 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:22.075182 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c129e-7414-41e1-8ade-fb0d02c7d9e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:22.076780 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c129e-7414-41e1-8ade-fb0d02c7d9e2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:22.083414 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 389/780] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:42:21 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:22.189428 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9c1ad-1eae-4bcf-82af-389094498e5d None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:22.190078 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9c1ad-1eae-4bcf-82af-389094498e5d None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:22.190516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9c1ad-1eae-4bcf-82af-389094498e5d None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:22.192367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 392/781] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:22.197815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde32a2-eec3-43a9-8708-33b2a76ab4ad None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:22.198120 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde32a2-eec3-43a9-8708-33b2a76ab4ad None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:22.198389 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde32a2-eec3-43a9-8708-33b2a76ab4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:22.220157 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-abde32a2-eec3-43a9-8708-33b2a76ab4ad None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:22.221442 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-abde32a2-eec3-43a9-8708-33b2a76ab4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:22.318030 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 390/782] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:22.346183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9c1ad-1eae-4bcf-82af-389094498e5d None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:22.440162 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4d64-3af7-40b4-9e51-eaa9b06f64fe None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:22.440498 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4d64-3af7-40b4-9e51-eaa9b06f64fe None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:22.440808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4d64-3af7-40b4-9e51-eaa9b06f64fe None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:22.445830 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 391/783] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:22.448191 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e4bf8-439b-46f3-bf7d-7fc203c883e3 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:22.448706 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e4bf8-439b-46f3-bf7d-7fc203c883e3 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:22.449238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e4bf8-439b-46f3-bf7d-7fc203c883e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:22.483143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-339e4bf8-439b-46f3-bf7d-7fc203c883e3 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:22.484695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-339e4bf8-439b-46f3-bf7d-7fc203c883e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:22.565809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 392/784] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:22.593921 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e4bf8-439b-46f3-bf7d-7fc203c883e3 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:22.721736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9c1ad-1eae-4bcf-82af-389094498e5d None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:22.743769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e4bf8-439b-46f3-bf7d-7fc203c883e3 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:22.814063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b36a5-83f5-4312-813e-c59920529b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:22.814892 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b36a5-83f5-4312-813e-c59920529b87 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:22.815184 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b36a5-83f5-4312-813e-c59920529b87 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:22.815469 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b36a5-83f5-4312-813e-c59920529b87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:22.817451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3c21e-d936-496e-85bc-3bcc76acbf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:22.818691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3c21e-d936-496e-85bc-3bcc76acbf9f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:22.818983 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3c21e-d936-496e-85bc-3bcc76acbf9f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:22.819265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3c21e-d936-496e-85bc-3bcc76acbf9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:22.846471 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b36a5-83f5-4312-813e-c59920529b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:22.850125 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3c21e-d936-496e-85bc-3bcc76acbf9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-66e3c21e-d936-496e-85bc-3bcc76acbf9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:22.863393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:22.864532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:22.865527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-e67b36a5-83f5-4312-813e-c59920529b87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:22.866233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:22.868637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:22.870837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:22.872395 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 393/785] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 519 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:22.872552 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 393/786] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:22.875811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3c21e-d936-496e-85bc-3bcc76acbf9f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:22.880299 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b36a5-83f5-4312-813e-c59920529b87 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.051777 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3c21e-d936-496e-85bc-3bcc76acbf9f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.086079 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.086763 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.087055 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.087338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.088104 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:23.088416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.089819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.099458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 394/787] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:23.104834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.164750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b36a5-83f5-4312-813e-c59920529b87 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.209283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a1511-a83f-4935-b628-0ffe73e3e65d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.244588 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.245419 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.245736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.245994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.257281 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.258681 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.258957 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.259376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.259668 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.259954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.260723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:23.261058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.262443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:23.276657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:23.285238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:23.286228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:23.286228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:23.286228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:23.286228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:23.286228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 395/788] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:23.286546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 394/789] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:23.286624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.292797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.391578 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82357c-2a92-4f89-8d80-acb2c463c358 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.428289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429e904-b10f-462b-8dfd-de7895e34080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.428984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429e904-b10f-462b-8dfd-de7895e34080 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.429306 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429e904-b10f-462b-8dfd-de7895e34080 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.429591 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429e904-b10f-462b-8dfd-de7895e34080 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.430345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429e904-b10f-462b-8dfd-de7895e34080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:23.430658 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429e904-b10f-462b-8dfd-de7895e34080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.432052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429e904-b10f-462b-8dfd-de7895e34080 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.445115 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 396/790] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:23.446420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b876a-4a81-4f9f-8ad5-b04867c45ca8 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.446715 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b876a-4a81-4f9f-8ad5-b04867c45ca8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.446994 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b876a-4a81-4f9f-8ad5-b04867c45ca8 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.452752 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 397/791] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:23.453564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b876a-4a81-4f9f-8ad5-b04867c45ca8 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.471002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9f7ef-f336-4b97-843c-6ef90b3362f8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.510545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561c7c4-9cc2-4892-acca-9dd14458eab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.511359 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561c7c4-9cc2-4892-acca-9dd14458eab3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.511811 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561c7c4-9cc2-4892-acca-9dd14458eab3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.512104 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561c7c4-9cc2-4892-acca-9dd14458eab3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.528446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561c7c4-9cc2-4892-acca-9dd14458eab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.530637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2561c7c4-9cc2-4892-acca-9dd14458eab3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-2561c7c4-9cc2-4892-acca-9dd14458eab3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:23.553537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:23.555082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:23.556972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:23.556972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:23.556972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:23.556972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:23.556972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 395/792] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:23.556972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25835c7-6364-4cca-92e2-bca11f73e1e2 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.557697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25835c7-6364-4cca-92e2-bca11f73e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.557697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25835c7-6364-4cca-92e2-bca11f73e1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.576381 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b876a-4a81-4f9f-8ad5-b04867c45ca8 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.577927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-c25835c7-6364-4cca-92e2-bca11f73e1e2 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:23.580119 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-c25835c7-6364-4cca-92e2-bca11f73e1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:23.619497 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.620611 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.621143 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.621628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] PATH_INFO: `/v3/users/nova` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.630035 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.631720 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:23.647442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:23.648527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:23.649719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:23.653155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 398/793] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:23.654216 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.657733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 396/794] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:23.659269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25835c7-6364-4cca-92e2-bca11f73e1e2 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.755675 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2421d-69c5-4e1c-9bf6-0fdea599155e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.757204 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25835c7-6364-4cca-92e2-bca11f73e1e2 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.790788 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.791468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.792047 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.792163 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.792427 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.792711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.793120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.793594 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:23.793792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.793896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.794818 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:23.795335 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.795449 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.796992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.813230 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 399/796] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:23.814412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.817600 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 397/796] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:23.818521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.917619 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710b64-5b2f-4baf-aad9-bf9304837fa9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.918139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3138b-744f-4e12-ab7c-9b80e3e91574 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:23.952084 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.952895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.953214 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.953495 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] PATH_INFO: `/v3/users/swift` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.954110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090a8ca-6783-453b-8b55-53b45dd3b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:23.955183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090a8ca-6783-453b-8b55-53b45dd3b14d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:23.955718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090a8ca-6783-453b-8b55-53b45dd3b14d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:23.956198 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090a8ca-6783-453b-8b55-53b45dd3b14d None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:23.961615 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:23.963081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:23.968175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090a8ca-6783-453b-8b55-53b45dd3b14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b090a8ca-6783-453b-8b55-53b45dd3b14d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:23.976479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:23.977644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:23.978744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:23.979456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:23.980421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:23.981504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:23.983798 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 400/798] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:23.984768 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090a8ca-6783-453b-8b55-53b45dd3b14d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:23.985361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 398/798] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:23.986242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.087510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72b880-8d26-47e7-a7ed-9daf4b41a25a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.088095 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090a8ca-6783-453b-8b55-53b45dd3b14d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.123349 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66711d60-7fb9-4c9e-ad2d-5ae8370dc806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.123587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.124044 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.124334 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.124619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.124914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66711d60-7fb9-4c9e-ad2d-5ae8370dc806 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.125387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66711d60-7fb9-4c9e-ad2d-5ae8370dc806 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.125519 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:24.125833 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.126100 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66711d60-7fb9-4c9e-ad2d-5ae8370dc806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.127279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.150160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 399/799] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:24.151110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.196680 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66711d60-7fb9-4c9e-ad2d-5ae8370dc806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.198431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66711d60-7fb9-4c9e-ad2d-5ae8370dc806 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.257383 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d756c-e6fa-4e89-a94f-a16e9089d3a8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.291926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.292610 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.292898 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.293214 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.293979 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:24.294295 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.295675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.304805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 400/800] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:24.305265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 401/801] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:42:24.310020 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.310800 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66711d60-7fb9-4c9e-ad2d-5ae8370dc806 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.411736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66711d60-7fb9-4c9e-ad2d-5ae8370dc806 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.412010 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1d646-040b-4d5e-b3f6-876faa8ab01c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.449701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.450554 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.450836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.451121 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.451317 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.452265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.452699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.453191 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.466617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.468101 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.469984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.471594 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:24.490345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:24.491621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:24.492792 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:24.492792 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:24.492792 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:24.492792 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:24.494737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:24.495792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:24.496795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:24.496795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:24.496795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:24.496795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:24.497107 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 401/802] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:24.497545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.499183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 402/803] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:24.500352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.605168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8b2ed-8fa6-41e6-852a-85cec0da8bc2 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.626037 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbad765-5c28-449f-9690-23f6ffc662f6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.640256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768eca2-b471-430c-b23b-36bc403be9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.641090 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768eca2-b471-430c-b23b-36bc403be9bf None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.641385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768eca2-b471-430c-b23b-36bc403be9bf None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.641669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768eca2-b471-430c-b23b-36bc403be9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.652194 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768eca2-b471-430c-b23b-36bc403be9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.653686 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768eca2-b471-430c-b23b-36bc403be9bf None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.659931 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.660606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.660889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.661208 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.661948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:24.662259 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.663604 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5768eca2-b471-430c-b23b-36bc403be9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:24.672369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:24.673485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:24.674674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:24.674674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:24.674674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:24.674674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:24.675060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 403/804] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:24.676356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.677601 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 402/805] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:24.679088 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b41d-b7a4-4ff6-8a00-c818cb8d962c None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.679380 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b41d-b7a4-4ff6-8a00-c818cb8d962c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.679666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848b41d-b7a4-4ff6-8a00-c818cb8d962c None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.685107 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 403/806] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:24.688614 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848b41d-b7a4-4ff6-8a00-c818cb8d962c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.780841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1e22-0aee-4a79-bef3-0b3880d54481 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.788143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848b41d-b7a4-4ff6-8a00-c818cb8d962c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.812444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1e078-a364-4b64-830a-c9294baa3c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.813129 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1e078-a364-4b64-830a-c9294baa3c25 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.813417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1e078-a364-4b64-830a-c9294baa3c25 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.813696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1e078-a364-4b64-830a-c9294baa3c25 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.814420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1e078-a364-4b64-830a-c9294baa3c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:24.814731 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1e078-a364-4b64-830a-c9294baa3c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.816131 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1e078-a364-4b64-830a-c9294baa3c25 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.819840 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b7201-e1ba-4856-aa38-c572fa6bda38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.820501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b7201-e1ba-4856-aa38-c572fa6bda38 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.820791 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b7201-e1ba-4856-aa38-c572fa6bda38 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.821096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b7201-e1ba-4856-aa38-c572fa6bda38 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.821825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b7201-e1ba-4856-aa38-c572fa6bda38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:24.822140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b7201-e1ba-4856-aa38-c572fa6bda38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.823491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b7201-e1ba-4856-aa38-c572fa6bda38 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.827436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 404/807] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:24.828746 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd30add-497c-4c00-9ff5-21663c7515f0 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.829063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd30add-497c-4c00-9ff5-21663c7515f0 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.829350 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd30add-497c-4c00-9ff5-21663c7515f0 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.833127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 405/808] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:24.833127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd30add-497c-4c00-9ff5-21663c7515f0 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.842756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 404/809] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:24.842756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16343431-7da3-494e-a8df-95f1662dcb20 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.842756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16343431-7da3-494e-a8df-95f1662dcb20 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.842756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16343431-7da3-494e-a8df-95f1662dcb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.858535 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-16343431-7da3-494e-a8df-95f1662dcb20 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:24.859771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-16343431-7da3-494e-a8df-95f1662dcb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:24.923707 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 405/810] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:24.924670 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16343431-7da3-494e-a8df-95f1662dcb20 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:24.930050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd30add-497c-4c00-9ff5-21663c7515f0 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:24.963240 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02500357-9d72-4ec1-a51d-529fe24cea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:24.964038 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02500357-9d72-4ec1-a51d-529fe24cea06 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.964326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02500357-9d72-4ec1-a51d-529fe24cea06 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.964607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02500357-9d72-4ec1-a51d-529fe24cea06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:24.971315 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02500357-9d72-4ec1-a51d-529fe24cea06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:24.973636 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02500357-9d72-4ec1-a51d-529fe24cea06 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-02500357-9d72-4ec1-a51d-529fe24cea06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:24.987071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:24.988316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:24.989704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:24.989704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:24.989704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:24.989704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:24.989704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:24.989704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:24.989704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:24.989704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:24.992947 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 406/811] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:24.994178 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccaa8-218d-476f-998f-c6ddb0812f50 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:24.994474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccaa8-218d-476f-998f-c6ddb0812f50 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:24.994760 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ccaa8-218d-476f-998f-c6ddb0812f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.010536 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-996ccaa8-218d-476f-998f-c6ddb0812f50 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:25.011727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-996ccaa8-218d-476f-998f-c6ddb0812f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:25.027780 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16343431-7da3-494e-a8df-95f1662dcb20 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.060192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.060947 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.061275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.061563 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.068420 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.069893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.074974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 407/812] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:25.075795 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ccaa8-218d-476f-998f-c6ddb0812f50 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:25.085110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:25.086176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 406/813] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:42:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:25.087239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.173896 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ccaa8-218d-476f-998f-c6ddb0812f50 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.184621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946cc19-50ac-4e4f-a202-dcda9aef0b8b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.207778 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.208565 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.208854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.209168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.215706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.217159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.217658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.218478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.218769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.219060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] PATH_INFO: `/v3/services/network` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.219706 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.221092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:25.230847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:25.231867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:25.232881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 408/814] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:25.233477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:25.234526 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 09 15:42:25.236531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:25.239231 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.239354 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 407/815] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/services/network => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:25.239354 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.329367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801f7b-701a-41cd-b87d-a800a2ad71fe None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.330673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19813fa5-48be-419c-8d9a-d1750b25d96c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.363509 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.364179 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.364468 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.364752 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.365595 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:25.365903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.367348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.367470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.368049 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.368370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.368653 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.369536 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:25.369853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.371283 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.379768 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 409/816] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:25.380570 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.393306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 408/817] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:25.393306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.475057 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc4165-0fc8-4ab5-b09b-5099573dfb42 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.485966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaab7f-793d-4a69-a749-a0b28e4feae9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.508006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.508813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.509135 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.509423 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.510093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.511460 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.519368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.520051 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.520337 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.520621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.521438 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:25.521749 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.523155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:25.523499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:25.524708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:25.525887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 410/818] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:25.526869 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.537180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 409/819] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:25.537965 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.630187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a9874-292a-4ebb-ae0c-34cfb3d5ea30 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.638012 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d0f1a-3878-447e-b813-2861cfa7026f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.664167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.664839 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.665169 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.665453 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.666172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] RBAC: Adding query filter params (name=network) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:25.666501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.667874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.672328 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 411/820] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:25.673134 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.677688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9fdc6-34a2-47c2-b04a-669c78f1e48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.678603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9fdc6-34a2-47c2-b04a-669c78f1e48b None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.678894 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9fdc6-34a2-47c2-b04a-669c78f1e48b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.679179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9fdc6-34a2-47c2-b04a-669c78f1e48b None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/48c22f093f45490e996d673b027bf391/roles/ebff922955b74c919d58613875326151` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.681205 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9fdc6-34a2-47c2-b04a-669c78f1e48b None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=48c22f093f45490e996d673b027bf391, role_id=ebff922955b74c919d58613875326151)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.682650 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9fdc6-34a2-47c2-b04a-669c78f1e48b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.695703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 410/821] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:25 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/48c22f093f45490e996d673b027bf391/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:42:25.696507 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9fdc6-34a2-47c2-b04a-669c78f1e48b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.780603 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e1c29-3671-4149-a92f-d601c9392524 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.801374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9fdc6-34a2-47c2-b04a-669c78f1e48b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.817511 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.818337 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.818635 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.818923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.830651 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.832193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.836679 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:25.837658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:25.837810 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:25.837995 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:25.838701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:25.839020 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:25.840429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:25.850379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 411/822] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:25.850379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:25.851264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:25.852356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:25.854130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:25.854130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:25.854130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:25.854130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:25.857001 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 412/823] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:25.857863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:25.955939 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6470abf-6a66-4e4a-aec7-dcfcaad22eb7 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:25.965160 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee286eea-f48f-4ca1-af07-d45bceb9a1cd None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:26.000470 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.000938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.001211 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.001512 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.001638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.001840 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:26.001951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.002195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:26.002507 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] RBAC: Adding query filter params (type=network) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:26.002822 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:26.003065 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=ef8546c698f9455cb81264bcee0d2344) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:26.003357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:26.004242 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:26.004786 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:26.014709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 413/824] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:26.014709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:26.015086 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 412/825] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:25 2022] GET /identity/v3/role_assignments?user.id=ef8546c698f9455cb81264bcee0d2344&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=ebff922955b74c919d58613875326151 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:26.015086 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:26.134305 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc6a42-16eb-41dc-ad3d-95471d7550f2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:26.141701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4001646-60de-4bf2-bd12-dd5182c14fc2 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:26.172392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.173118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.173418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.173706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:26.174488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:26.174770 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:26.176812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:26.181599 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.182633 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.183151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.183650 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:26.184624 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:26.185191 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:26.186813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:26.187277 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 414/826] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 9 15:42:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:26.188242 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:26.201639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 413/827] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:26.202893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:26.303905 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894d5fb-3b36-405d-b8ae-f1a51700b6b5 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:26.332863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7585a-2e5e-4730-9a42-b1090d752f23 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:26.349291 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e0ac-d44a-4f99-8bef-de015b346d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.350337 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e0ac-d44a-4f99-8bef-de015b346d94 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.350858 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e0ac-d44a-4f99-8bef-de015b346d94 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.351355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e0ac-d44a-4f99-8bef-de015b346d94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:26.352314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996e0ac-d44a-4f99-8bef-de015b346d94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:26.353936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996e0ac-d44a-4f99-8bef-de015b346d94 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:26.371669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d177d1-b260-4a46-ac7a-e13433c0f96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.372813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d177d1-b260-4a46-ac7a-e13433c0f96a None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.373369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d177d1-b260-4a46-ac7a-e13433c0f96a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.373983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d177d1-b260-4a46-ac7a-e13433c0f96a None admin] PATH_INFO: `/v3/endpoints` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:26.374934 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d177d1-b260-4a46-ac7a-e13433c0f96a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:26.381077 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 415/828] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:42:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:26.381077 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e0ac-d44a-4f99-8bef-de015b346d94 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:26.381915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d177d1-b260-4a46-ac7a-e13433c0f96a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:26.426719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 414/829] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:42:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:26.545030 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e0ac-d44a-4f99-8bef-de015b346d94 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:26.589067 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.590324 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.590836 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.591345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:26.600282 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:26.602169 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:26.617866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:26.618932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:26.619921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:26.619921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:26.619921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:26.619921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:26.619921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:26.619921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:26.619921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:26.619921 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:26.622647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 416/830] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:26.628837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d177d1-b260-4a46-ac7a-e13433c0f96a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:26.768661 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d177d1-b260-4a46-ac7a-e13433c0f96a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:26.802809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a54bb-c621-4bc7-8ba3-3fc22161e7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.803870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a54bb-c621-4bc7-8ba3-3fc22161e7e9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.804393 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a54bb-c621-4bc7-8ba3-3fc22161e7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.804895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a54bb-c621-4bc7-8ba3-3fc22161e7e9 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:26.805942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a54bb-c621-4bc7-8ba3-3fc22161e7e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:26.806480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a54bb-c621-4bc7-8ba3-3fc22161e7e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:26.808132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a54bb-c621-4bc7-8ba3-3fc22161e7e9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:26.821762 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 415/831] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:26.827322 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:26.946821 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170216e7-f29a-4ea7-9feb-ebf32d2090b1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:26.982583 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164e474-94b0-47d0-a92c-9fab58c4afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:26.984324 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164e474-94b0-47d0-a92c-9fab58c4afb4 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:26.984827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164e474-94b0-47d0-a92c-9fab58c4afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:26.985356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164e474-94b0-47d0-a92c-9fab58c4afb4 None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/6cf6a2d926e9494abb63a9535b547b6e/roles/d28cdc73a0fc42f6b6e0c45768d418ae` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:27.001066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164e474-94b0-47d0-a92c-9fab58c4afb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=6cf6a2d926e9494abb63a9535b547b6e, role_id=d28cdc73a0fc42f6b6e0c45768d418ae)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:27.002830 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164e474-94b0-47d0-a92c-9fab58c4afb4 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:27.017856 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 417/832] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:26 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/6cf6a2d926e9494abb63a9535b547b6e/roles/d28cdc73a0fc42f6b6e0c45768d418ae => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:42:27.990530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376a4fb-463a-407f-8fb9-bca7266c4417 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:27.991527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376a4fb-463a-407f-8fb9-bca7266c4417 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:27.992036 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376a4fb-463a-407f-8fb9-bca7266c4417 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:27.993909 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 416/833] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:27.999025 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b4ca-9afe-409a-866e-ddaa5b4aac4a None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:27.999444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b4ca-9afe-409a-866e-ddaa5b4aac4a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:27.999640 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b4ca-9afe-409a-866e-ddaa5b4aac4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.017219 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-e388b4ca-9afe-409a-866e-ddaa5b4aac4a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:28.018426 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-e388b4ca-9afe-409a-866e-ddaa5b4aac4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:28.087808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 418/834] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:28.111285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376a4fb-463a-407f-8fb9-bca7266c4417 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:28.218889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376a4fb-463a-407f-8fb9-bca7266c4417 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:28.256502 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf343be0-98f7-4982-a5df-2343010bc2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:28.257347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf343be0-98f7-4982-a5df-2343010bc2b1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.257637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf343be0-98f7-4982-a5df-2343010bc2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.257920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf343be0-98f7-4982-a5df-2343010bc2b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.268382 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf343be0-98f7-4982-a5df-2343010bc2b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-bf343be0-98f7-4982-a5df-2343010bc2b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:28.276087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:28.277224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:28.278181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 417/835] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:28.283524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b4ca-9afe-409a-866e-ddaa5b4aac4a None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:28.384054 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b4ca-9afe-409a-866e-ddaa5b4aac4a None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:28.401106 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad67d-409b-4caa-9f02-265ae5915bec None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.401411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad67d-409b-4caa-9f02-265ae5915bec None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.401980 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad67d-409b-4caa-9f02-265ae5915bec None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.403622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 418/836] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:28.410641 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab3e1-013f-48f8-9ffc-f255fac29d3c None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.411094 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab3e1-013f-48f8-9ffc-f255fac29d3c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.411385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab3e1-013f-48f8-9ffc-f255fac29d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.419321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:28.420001 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.420289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.420567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.421363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:28.421677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:28.423055 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:28.432154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 419/837] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:28.433644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-391ab3e1-013f-48f8-9ffc-f255fac29d3c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:28.435061 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-391ab3e1-013f-48f8-9ffc-f255fac29d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:28.437572 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:28.529155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 419/838] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:28.547560 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ab3e1-013f-48f8-9ffc-f255fac29d3c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:28.557448 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9c6d-5d79-4a09-b158-9dee866bc3b8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:28.600320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:28.601736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.602032 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.602321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.615971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:28.617474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:28.634779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:28.635826 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:28.637616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:28.637616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:28.637616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:28.637616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:28.640772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 420/839] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:28.641863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:28.677641 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ab3e1-013f-48f8-9ffc-f255fac29d3c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:28.725207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:28.726346 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.727159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.727613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.728764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:28.730649 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:28.747249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:28.748533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:28.749724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:28.757462 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078b73d-6dfc-485b-ae57-31a034ba4fdd None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:28.757625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 420/840] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:28.762167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:28.791862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:28.792465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.792806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.793143 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.793940 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:28.794280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:28.795762 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:28.813353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 421/841] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:28.816136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:28.887331 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba75dd-c4fa-4ae4-bce1-3a981ea8d339 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:28.923748 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b6541-b6de-4a57-8648-d0f5273eeb75 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:28.925785 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:28.926738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.927249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.927748 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.928690 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:28.929284 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:28.930913 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:28.941195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 421/842] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:28.942726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:28.960187 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:28.961063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:28.961356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:28.961641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:28.969663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:28.971206 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:28.991537 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:28.992678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 09 15:42:28.994399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:28.997482 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 422/843] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:28.998499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.051980 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8ccf-17ac-4c8b-b7f9-285f3c44af19 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.085780 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a151b45-d878-4179-b624-9d1072d90536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.086453 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a151b45-d878-4179-b624-9d1072d90536 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.086743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a151b45-d878-4179-b624-9d1072d90536 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.087023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a151b45-d878-4179-b624-9d1072d90536 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.087763 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a151b45-d878-4179-b624-9d1072d90536 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:29.088082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a151b45-d878-4179-b624-9d1072d90536 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:29.089489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a151b45-d878-4179-b624-9d1072d90536 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:29.094661 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 422/844] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:29.095840 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6b4b7-c3bd-4e93-a843-4a5c354df504 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.096130 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6b4b7-c3bd-4e93-a843-4a5c354df504 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.096409 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6b4b7-c3bd-4e93-a843-4a5c354df504 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.098225 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 423/845] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:29.104183 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b698b08-295e-4934-aff8-6392d1b6b84a None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.104430 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b698b08-295e-4934-aff8-6392d1b6b84a None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.104721 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b698b08-295e-4934-aff8-6392d1b6b84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.109410 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b98bd-65ea-471b-8c64-a006307999ef None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.123520 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-0b698b08-295e-4934-aff8-6392d1b6b84a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:29.124817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-0b698b08-295e-4934-aff8-6392d1b6b84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:29.154710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.155406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.155699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.155978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.156842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:29.157215 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:29.158682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:29.182992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 423/846] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 9 15:42:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:29.186260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.209698 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 424/847] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:29.232636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b698b08-295e-4934-aff8-6392d1b6b84a None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.319905 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c3767-37b0-4401-95b4-2aa85bfd73f9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.341626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b698b08-295e-4934-aff8-6392d1b6b84a None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.355972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6c400-7edf-4d39-8f8c-920f7ac6e8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.356918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6c400-7edf-4d39-8f8c-920f7ac6e8bf None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.357244 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6c400-7edf-4d39-8f8c-920f7ac6e8bf None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.357529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6c400-7edf-4d39-8f8c-920f7ac6e8bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.375947 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3f91a-d7cd-45f5-8917-ac1d65d05017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.376759 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f91a-d7cd-45f5-8917-ac1d65d05017 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.377062 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f91a-d7cd-45f5-8917-ac1d65d05017 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.377369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f91a-d7cd-45f5-8917-ac1d65d05017 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.378667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3f91a-d7cd-45f5-8917-ac1d65d05017 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:29.380101 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3f91a-d7cd-45f5-8917-ac1d65d05017 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-93f3f91a-d7cd-45f5-8917-ac1d65d05017 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:29.392016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:29.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:29.394885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 425/848] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:29.395963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702cad2-6a29-4350-b786-cac379d62f87 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.395963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702cad2-6a29-4350-b786-cac379d62f87 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.395963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702cad2-6a29-4350-b786-cac379d62f87 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.396328 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 426/849] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:29.401148 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702cad2-6a29-4350-b786-cac379d62f87 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.430298 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6c400-7edf-4d39-8f8c-920f7ac6e8bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:29.431757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6c400-7edf-4d39-8f8c-920f7ac6e8bf None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:29.499533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702cad2-6a29-4350-b786-cac379d62f87 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.534735 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.535414 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.535705 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.535993 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.536513 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 424/850] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:42:29.536922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:29.537273 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:29.537895 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17e43d-2f2b-48f4-a0fa-227a2bed0c94 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.538193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17e43d-2f2b-48f4-a0fa-227a2bed0c94 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.538481 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17e43d-2f2b-48f4-a0fa-227a2bed0c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.538666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:29.547739 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 427/851] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:29.547739 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.561439 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-6b17e43d-2f2b-48f4-a0fa-227a2bed0c94 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:29.563100 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-6b17e43d-2f2b-48f4-a0fa-227a2bed0c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:29.633473 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 425/852] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:29.634252 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17e43d-2f2b-48f4-a0fa-227a2bed0c94 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.655337 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b9861-cd25-49b9-9482-1e7f106660f8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.691361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.692226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.692517 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.692856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.703817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:29.705375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:29.727168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:29.728223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:29.729279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:29.729279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:29.729279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:29.729279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:29.730110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 428/853] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:29.731029 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.738076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17e43d-2f2b-48f4-a0fa-227a2bed0c94 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.771452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.772186 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.772478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.772770 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.773639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:29.773964 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:29.775505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:29.780684 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 426/854] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:29.781532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.846169 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0074355-23e9-4411-9b45-eebd6dd567cf None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.885089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92bdad-1326-4150-9fb9-92d222ee1767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.886305 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92bdad-1326-4150-9fb9-92d222ee1767 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.886855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92bdad-1326-4150-9fb9-92d222ee1767 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.887376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92bdad-1326-4150-9fb9-92d222ee1767 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.901273 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92bdad-1326-4150-9fb9-92d222ee1767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-3e92bdad-1326-4150-9fb9-92d222ee1767 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:29.910269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:29.911424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:42:29.912368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:29.915476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 429/855] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:29.921247 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92bdad-1326-4150-9fb9-92d222ee1767 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:29.924048 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e6414-d24f-46ac-a21e-4f4313412ddb None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:29.963117 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:29.964358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:29.964892 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:29.965424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:29.966399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:29.966999 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:29.968692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:29.983482 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 427/856] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:29.989842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:30.049808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92bdad-1326-4150-9fb9-92d222ee1767 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.085381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.086876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.087405 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.087916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:30.088902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:30.089451 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:30.091040 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:30.100783 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 430/857] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:30.100946 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546ad91-0846-4621-9ba8-d7db242dcb1a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.105860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:30.135348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.136112 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.136407 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.136690 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:30.144928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:30.145692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:30.160954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:30.161996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:30.163018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 428/858] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:30.167613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:30.215004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662d549-b2b0-47bc-88b5-3d839518720f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.250974 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.252216 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.252750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.253361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:30.265244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:30.267134 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:30.272444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8e7cf-b7cc-474b-a9b9-fee42667c39e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:30.286209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:30.287234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:30.288257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:30.288257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:30.288257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:30.288257 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:30.290523 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 431/859] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:30.295676 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:30.312485 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.313102 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.313397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.313682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:30.314541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:30.314872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:30.316392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:30.334569 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 429/860] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:30.335482 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:30.405853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe395ea-a659-40c1-abee-67c049905e95 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.440182 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.441869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.442378 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.442888 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:30.443837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:30.444376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:30.446007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:30.458925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 432/861] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:30.464841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ae565-0bd0-4640-976d-0614c26976b2 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.466939 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:30.512149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.512869 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.513194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.513474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:30.518111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=48c22f093f45490e996d673b027bf391) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:30.519744 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:30.522086 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:30.530972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 430/862] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:30 2022] GET /identity/v3/role_assignments?user.id=48c22f093f45490e996d673b027bf391&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=ebff922955b74c919d58613875326151 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:30.595213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84e62-dfbb-4068-b0d0-fa9f0e38cd74 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.634634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.635994 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.636509 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.637023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:30.646333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:30.648166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:30.665149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:30.666275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:30.667375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:30.670347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 433/863] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:42:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:30.675979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:30.783490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf680-b9c2-439f-ac57-80c5a364db1e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.817785 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.818722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.819188 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.819638 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:30.820720 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:30.821296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:30.822956 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:30.841563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 431/864] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:42:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:30.848180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:30.950422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096ad4c-ce81-4b79-bcba-fc485807df71 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:30.987735 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b09ae0-1627-46da-b911-cefe1230c0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:30.988430 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b09ae0-1627-46da-b911-cefe1230c0c5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:30.988722 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b09ae0-1627-46da-b911-cefe1230c0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:30.989005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b09ae0-1627-46da-b911-cefe1230c0c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:31.059864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b09ae0-1627-46da-b911-cefe1230c0c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:31.061366 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b09ae0-1627-46da-b911-cefe1230c0c5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:31.164239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 434/865] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:42:31.170498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:31.278141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e15cae-0c2b-4096-bd2c-33d1618f3390 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:31.312005 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b26c-730b-4211-ad3b-c83c49423771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:31.312813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b26c-730b-4211-ad3b-c83c49423771 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:31.313134 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b26c-730b-4211-ad3b-c83c49423771 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:31.313418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b26c-730b-4211-ad3b-c83c49423771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:31.324238 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b26c-730b-4211-ad3b-c83c49423771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:31.325805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b26c-730b-4211-ad3b-c83c49423771 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-8989b26c-730b-4211-ad3b-c83c49423771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:31.343942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:31.344973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:31.346058 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:31.346058 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:31.346058 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:31.346058 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:31.346058 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 432/866] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:31.354154 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b09ae0-1627-46da-b911-cefe1230c0c5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:31.388778 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabadb-a162-4a05-b245-4fd043f10454 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:31.389141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabadb-a162-4a05-b245-4fd043f10454 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:31.390095 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabadb-a162-4a05-b245-4fd043f10454 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:31.390888 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 433/867] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:31.396963 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3ce3a-a7c3-4823-a3d1-1e7170c4b08c None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:31.397283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3ce3a-a7c3-4823-a3d1-1e7170c4b08c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:31.397569 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3ce3a-a7c3-4823-a3d1-1e7170c4b08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:31.419007 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-76d3ce3a-a7c3-4823-a3d1-1e7170c4b08c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:31.420805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-76d3ce3a-a7c3-4823-a3d1-1e7170c4b08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:31.472760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b09ae0-1627-46da-b911-cefe1230c0c5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:31.509075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 434/868] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:31.521062 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:31.521808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:31.522103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:31.522447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:31.523190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:31.523508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:31.524896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:31.528518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3ce3a-a7c3-4823-a3d1-1e7170c4b08c None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:31.539606 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 435/869] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:31.549176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:31.647484 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3ce3a-a7c3-4823-a3d1-1e7170c4b08c None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:31.668199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc797b1f-47fa-45ca-b175-008883f4987d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:31.692312 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb27b8-843f-4390-b5b1-92da9b037820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:31.693359 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb27b8-843f-4390-b5b1-92da9b037820 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:31.693727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb27b8-843f-4390-b5b1-92da9b037820 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:31.694110 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb27b8-843f-4390-b5b1-92da9b037820 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:31.695106 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb27b8-843f-4390-b5b1-92da9b037820 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:31.696973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb27b8-843f-4390-b5b1-92da9b037820 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:31.705067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:31.706238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:31.706697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:31.707151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:31.714525 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 435/870] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:42:31 2022] POST /identity/v3/services => generated 269 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:42:31.715218 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:31.716712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:31.736884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:31.738613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:31.739619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 436/871] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:31.745283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb27b8-843f-4390-b5b1-92da9b037820 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:31.867640 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb27b8-843f-4390-b5b1-92da9b037820 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:31.904402 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8206de-d2ec-49af-92b7-cf49da006efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:31.905478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8206de-d2ec-49af-92b7-cf49da006efb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:31.905987 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8206de-d2ec-49af-92b7-cf49da006efb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:31.906479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8206de-d2ec-49af-92b7-cf49da006efb None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:31.907494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8206de-d2ec-49af-92b7-cf49da006efb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:31.908040 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8206de-d2ec-49af-92b7-cf49da006efb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:31.910280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8206de-d2ec-49af-92b7-cf49da006efb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:31.926283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 436/872] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:31.932054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:32.040710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cc641-3883-4c80-8a18-718c82f5dd51 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:32.076108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53ea90-05a1-4572-bbb4-17aeae141f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:32.077209 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ea90-05a1-4572-bbb4-17aeae141f4a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:32.077746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ea90-05a1-4572-bbb4-17aeae141f4a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:32.078243 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ea90-05a1-4572-bbb4-17aeae141f4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:32.079333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53ea90-05a1-4572-bbb4-17aeae141f4a None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=6cf6a2d926e9494abb63a9535b547b6e) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:32.079880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53ea90-05a1-4572-bbb4-17aeae141f4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:32.081614 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53ea90-05a1-4572-bbb4-17aeae141f4a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:32.088343 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 437/873] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:31 2022] GET /identity/v3/role_assignments?user.id=6cf6a2d926e9494abb63a9535b547b6e&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:32.131008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0bddc-2c16-4132-ad90-e1e78243995e None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:32.131399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0bddc-2c16-4132-ad90-e1e78243995e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:32.131573 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0bddc-2c16-4132-ad90-e1e78243995e None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:32.133167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 437/874] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:32.139035 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21869ce-0f65-493a-ac2d-cf6a340d820a None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:32.139335 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21869ce-0f65-493a-ac2d-cf6a340d820a None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:32.139626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21869ce-0f65-493a-ac2d-cf6a340d820a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:32.158269 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-c21869ce-0f65-493a-ac2d-cf6a340d820a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:32.160080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-c21869ce-0f65-493a-ac2d-cf6a340d820a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:32.235852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 438/875] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:32 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:32.259355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0bddc-2c16-4132-ad90-e1e78243995e None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:32.369273 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0bddc-2c16-4132-ad90-e1e78243995e None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:32.410613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1b3ec-1ecb-4ef3-bcca-c21c291e67e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:32.411671 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1b3ec-1ecb-4ef3-bcca-c21c291e67e4 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:32.412203 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1b3ec-1ecb-4ef3-bcca-c21c291e67e4 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:32.412697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1b3ec-1ecb-4ef3-bcca-c21c291e67e4 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:32.413692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1b3ec-1ecb-4ef3-bcca-c21c291e67e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:32.415411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1b3ec-1ecb-4ef3-bcca-c21c291e67e4 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:32.431412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 438/876] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:42:32 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:32.965531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000d472-35fc-4f60-ac5a-6b82a5daeeb5 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:32.965844 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000d472-35fc-4f60-ac5a-6b82a5daeeb5 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:32.966103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000d472-35fc-4f60-ac5a-6b82a5daeeb5 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:32.967769 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 439/877] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:32.973518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64856f1-4726-485a-a794-fb486badc997 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:32.973814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64856f1-4726-485a-a794-fb486badc997 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:32.974097 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64856f1-4726-485a-a794-fb486badc997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:32.992108 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-a64856f1-4726-485a-a794-fb486badc997 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:32.993330 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-a64856f1-4726-485a-a794-fb486badc997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:33.067563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 439/878] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:33.089364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000d472-35fc-4f60-ac5a-6b82a5daeeb5 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:33.190216 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000d472-35fc-4f60-ac5a-6b82a5daeeb5 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:33.224154 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:33.224965 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:33.225279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:33.225560 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:33.226214 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:33.227586 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:33.243714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:33.245348 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:33.246288 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:33.251942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 440/879] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:33.253916 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64856f1-4726-485a-a794-fb486badc997 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:33.354789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64856f1-4726-485a-a794-fb486badc997 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:33.392227 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8e1ca-35f3-4101-9ef7-b634b7c01cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:33.392901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e1ca-35f3-4101-9ef7-b634b7c01cb0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:33.393220 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e1ca-35f3-4101-9ef7-b634b7c01cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:33.393533 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e1ca-35f3-4101-9ef7-b634b7c01cb0 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:33.394259 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e1ca-35f3-4101-9ef7-b634b7c01cb0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:33.394568 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e1ca-35f3-4101-9ef7-b634b7c01cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:33.395948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e1ca-35f3-4101-9ef7-b634b7c01cb0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:33.405159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 440/880] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:33.406969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:33.511550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26847c-5561-4a4b-a7b7-cddf6a446be1 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:33.547516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e92169-ad7a-48f4-b94f-dc42ace0d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:33.548569 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e92169-ad7a-48f4-b94f-dc42ace0d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:33.549140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e92169-ad7a-48f4-b94f-dc42ace0d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:33.549682 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e92169-ad7a-48f4-b94f-dc42ace0d3d0 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:33.551151 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e92169-ad7a-48f4-b94f-dc42ace0d3d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:33.551685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e92169-ad7a-48f4-b94f-dc42ace0d3d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:33.553305 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e92169-ad7a-48f4-b94f-dc42ace0d3d0 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:33.558672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 441/881] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:34.085362 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368d312-5a56-4f37-8052-00ffcb3beabc None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.085654 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368d312-5a56-4f37-8052-00ffcb3beabc None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.086205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368d312-5a56-4f37-8052-00ffcb3beabc None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.088084 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 441/882] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:34.093684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe496-e672-4dea-8977-34b803261e4c None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.094234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe496-e672-4dea-8977-34b803261e4c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.094743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe496-e672-4dea-8977-34b803261e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.112539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-363fe496-e672-4dea-8977-34b803261e4c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:34.114104 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-363fe496-e672-4dea-8977-34b803261e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:34.185697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 442/883] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:34.209151 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368d312-5a56-4f37-8052-00ffcb3beabc None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:34.316758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368d312-5a56-4f37-8052-00ffcb3beabc None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:34.351966 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:34.352814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.353130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.353414 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.354100 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:34.355530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:34.369960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:34.371111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:34.372071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 442/884] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:42:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:34.380117 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fe496-e672-4dea-8977-34b803261e4c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:34.486771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fe496-e672-4dea-8977-34b803261e4c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:34.525997 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b836278-fd6c-4b5c-9d81-c3d0cfa558c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:34.526696 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836278-fd6c-4b5c-9d81-c3d0cfa558c5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.527132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836278-fd6c-4b5c-9d81-c3d0cfa558c5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.527417 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836278-fd6c-4b5c-9d81-c3d0cfa558c5 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.528147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b836278-fd6c-4b5c-9d81-c3d0cfa558c5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:34.528465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b836278-fd6c-4b5c-9d81-c3d0cfa558c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:34.529871 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b836278-fd6c-4b5c-9d81-c3d0cfa558c5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:34.535285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 443/885] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:34.541323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:34.563869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269936d-2727-41bf-a89e-98b5a5d9cb39 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.564186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269936d-2727-41bf-a89e-98b5a5d9cb39 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.564444 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269936d-2727-41bf-a89e-98b5a5d9cb39 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.566706 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 444/886] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:34.571658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53dd8-3d3b-4334-a4af-470965886eec None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.571954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53dd8-3d3b-4334-a4af-470965886eec None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.572245 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a53dd8-3d3b-4334-a4af-470965886eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.588946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-38a53dd8-3d3b-4334-a4af-470965886eec None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:34.590233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-38a53dd8-3d3b-4334-a4af-470965886eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:34.654564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee80f20-1102-4e4d-a420-093817acba6d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:34.669358 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 445/887] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:34.688278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a53dd8-3d3b-4334-a4af-470965886eec None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:34.695437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:34.696541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.697119 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.698276 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.699320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:34.699863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:34.702349 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:34.710293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 443/888] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:34.717221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:34.817918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a53dd8-3d3b-4334-a4af-470965886eec None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:34.831428 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9338e-b1ce-4294-b8e2-498cf5a137a9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:34.865854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abdc99-f14f-44a3-9b19-3d5b263df267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:34.867024 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abdc99-f14f-44a3-9b19-3d5b263df267 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.867529 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abdc99-f14f-44a3-9b19-3d5b263df267 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.867992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e3988-e035-42e9-9673-46cefdbb3650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:34.868268 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abdc99-f14f-44a3-9b19-3d5b263df267 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.869259 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e3988-e035-42e9-9673-46cefdbb3650 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.869624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e3988-e035-42e9-9673-46cefdbb3650 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.869980 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e3988-e035-42e9-9673-46cefdbb3650 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.870922 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e3988-e035-42e9-9673-46cefdbb3650 None admin] RBAC: Adding query filter params (interface=public, service_id=5582a846a8404d0cb595d867e4b59189, region_id=RegionOne) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:34.871243 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e3988-e035-42e9-9673-46cefdbb3650 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:34.872688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e3988-e035-42e9-9673-46cefdbb3650 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:34.879309 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 444/889] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:42:34 2022] GET /identity/v3/endpoints?service_id=5582a846a8404d0cb595d867e4b59189&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:34.880757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02045b2-e813-4c2c-878a-8be7ded1b7fd None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.881153 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02045b2-e813-4c2c-878a-8be7ded1b7fd None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.881506 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02045b2-e813-4c2c-878a-8be7ded1b7fd None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.883054 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 445/890] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:34.883319 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abdc99-f14f-44a3-9b19-3d5b263df267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:34.887279 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02045b2-e813-4c2c-878a-8be7ded1b7fd None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-51abdc99-f14f-44a3-9b19-3d5b263df267 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:34.891773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:34.892822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:34.894142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 446/891] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:42:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:34.894940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab778-81be-4b7b-ab67-9b1a62377063 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:34.894940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab778-81be-4b7b-ab67-9b1a62377063 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:34.895090 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ab778-81be-4b7b-ab67-9b1a62377063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:34.918045 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-d83ab778-81be-4b7b-ab67-9b1a62377063 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:34.919311 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-d83ab778-81be-4b7b-ab67-9b1a62377063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:34.988013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 447/892] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:34.989335 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ab778-81be-4b7b-ab67-9b1a62377063 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.006443 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02045b2-e813-4c2c-878a-8be7ded1b7fd None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.042092 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f22e3-514b-44d4-b26a-a837c0caa23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.042770 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f22e3-514b-44d4-b26a-a837c0caa23a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.043062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f22e3-514b-44d4-b26a-a837c0caa23a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.043349 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f22e3-514b-44d4-b26a-a837c0caa23a None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.044045 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f22e3-514b-44d4-b26a-a837c0caa23a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.045453 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f22e3-514b-44d4-b26a-a837c0caa23a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.061082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 446/893] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:42:34 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:42:35.061082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f22e3-514b-44d4-b26a-a837c0caa23a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.102805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ab778-81be-4b7b-ab67-9b1a62377063 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.139676 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.140372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.140661 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.140945 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.141754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:35.142067 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.143515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.152940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 448/894] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:42:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:35.158460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.185283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f22e3-514b-44d4-b26a-a837c0caa23a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.228584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.229835 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.230407 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.230934 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.231894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.233563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:35.247398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:35.248894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:35.250080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:35.253214 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 447/895] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:35.258763 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.292892 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e9502-c819-45a3-868d-14d0c8e11f00 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.337976 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.338805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.339095 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.339380 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.351500 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.353025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:35.370673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:35.371709 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:35.372729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:35.372729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:35.372729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:35.372729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:35.372729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 449/896] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:35.377881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.385296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf568c-fb67-4d2f-9b2d-5d98c7507443 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.431046 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.432115 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.433094 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.433630 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.434888 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:35.435595 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.437830 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.444801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 448/897] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:35.450914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.481885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348799b-312a-4805-a830-8ecc8040c874 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.516503 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.517393 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.517787 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.518171 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.519620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:35.520038 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.521611 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.539799 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 450/898] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:35.541143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.559658 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68507fc-33ca-40e4-8b2a-e9d9ce7bcd51 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.594972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.595996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.596531 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.597805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.598719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:35.599247 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.600853 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.606518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 449/899] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:35.611762 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.647444 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ee2ad-5767-4cf2-bd86-9d7ce3261adb None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.683497 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.684668 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.685166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.685629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.694830 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.696325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:35.716516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:35.717744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:35.718847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:35.721386 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 451/900] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:35.724322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.738012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daae7af-7941-4c0d-b770-9c187a3e983e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.783622 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.784968 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.785693 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.786528 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.787845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] RBAC: Adding query filter params (interface=public, service_id=9852f1c8c5eb4f4589a37f24187d802d, region_id=RegionOne) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:35.788397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.790184 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.797363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 450/901] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/endpoints?service_id=9852f1c8c5eb4f4589a37f24187d802d&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:35.802691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.838159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6042b-6c69-49f1-bb38-f61acc2fdbde None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.874379 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0929216-e9b3-4562-a2b9-df39227255e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.874990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929216-e9b3-4562-a2b9-df39227255e4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.875282 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929216-e9b3-4562-a2b9-df39227255e4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.875565 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929216-e9b3-4562-a2b9-df39227255e4 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.876454 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0929216-e9b3-4562-a2b9-df39227255e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:35.876771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0929216-e9b3-4562-a2b9-df39227255e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.878797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0929216-e9b3-4562-a2b9-df39227255e4 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.896141 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 452/902] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:35.897498 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdee4b-de68-4b0e-bac1-d7c0577aa8a7 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.897798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdee4b-de68-4b0e-bac1-d7c0577aa8a7 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.898080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdee4b-de68-4b0e-bac1-d7c0577aa8a7 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.903478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 453/903] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:35.903478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdee4b-de68-4b0e-bac1-d7c0577aa8a7 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:35.924707 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47b8ec-357b-4334-afd9-21a0251069ca None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:35.959608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bb683-7f62-4139-98db-2d22507f1bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:35.960286 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bb683-7f62-4139-98db-2d22507f1bac None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.960575 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bb683-7f62-4139-98db-2d22507f1bac None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.960856 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bb683-7f62-4139-98db-2d22507f1bac None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.961581 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bb683-7f62-4139-98db-2d22507f1bac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:35.962946 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bb683-7f62-4139-98db-2d22507f1bac None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:35.968807 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 451/904] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:42:35.977844 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5e45a-401c-4192-ba89-71a11f1861cd None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:35.978205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5e45a-401c-4192-ba89-71a11f1861cd None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:35.978415 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5e45a-401c-4192-ba89-71a11f1861cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:35.996619 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-ccd5e45a-401c-4192-ba89-71a11f1861cd None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:35.997893 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-ccd5e45a-401c-4192-ba89-71a11f1861cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:36.036544 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdee4b-de68-4b0e-bac1-d7c0577aa8a7 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:36.077559 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 452/905] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:36.084681 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a946f3-6727-4891-a2b4-ef50e454da1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:36.085716 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a946f3-6727-4891-a2b4-ef50e454da1b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:36.086193 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a946f3-6727-4891-a2b4-ef50e454da1b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:36.086689 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a946f3-6727-4891-a2b4-ef50e454da1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:36.098126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5e45a-401c-4192-ba89-71a11f1861cd None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:36.173642 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a946f3-6727-4891-a2b4-ef50e454da1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:36.176217 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a946f3-6727-4891-a2b4-ef50e454da1b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:36.216932 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5e45a-401c-4192-ba89-71a11f1861cd None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:36.252629 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d29d4-e84c-4637-a403-874325447839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:36.253897 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d29d4-e84c-4637-a403-874325447839 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:36.254195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d29d4-e84c-4637-a403-874325447839 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:36.254507 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d29d4-e84c-4637-a403-874325447839 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:36.255452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d29d4-e84c-4637-a403-874325447839 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:36.256634 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d29d4-e84c-4637-a403-874325447839 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:36.273114 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 453/906] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:42:36 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:36.315130 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 454/907] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:42:36.321699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d29d4-e84c-4637-a403-874325447839 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:36.442039 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d29d4-e84c-4637-a403-874325447839 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:36.478130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:36.478936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:36.479222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:36.479500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:36.494211 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:36.496655 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:36.514420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:36.515466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:36.516476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:36.516476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:36.516476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:36.516476 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:36.520408 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 454/908] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:36.522193 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a946f3-6727-4891-a2b4-ef50e454da1b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:36.626024 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a946f3-6727-4891-a2b4-ef50e454da1b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:36.662319 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:36.662994 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:36.663280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:36.663561 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:36.664293 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:36.664602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:36.665989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:36.677552 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 455/909] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:36.682976 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:36.784716 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f22aec-7683-40c5-84ad-6f4ce22447f3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:36.818527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:36.819337 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:36.819623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:36.819901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:36.827002 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:36.828450 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:36.843320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:36.844397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:36.845445 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 455/910] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:36.851088 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:36.954128 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d40cbf-19ff-4a30-9d99-6d7a8946989c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:36.988317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b12ab-ead9-4ab7-9322-9dde02dce1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:36.988999 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b12ab-ead9-4ab7-9322-9dde02dce1da None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:36.989316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b12ab-ead9-4ab7-9322-9dde02dce1da None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:36.989595 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b12ab-ead9-4ab7-9322-9dde02dce1da None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:36.990393 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b12ab-ead9-4ab7-9322-9dde02dce1da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:36.990714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b12ab-ead9-4ab7-9322-9dde02dce1da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:36.992137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b12ab-ead9-4ab7-9322-9dde02dce1da None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:37.004584 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 456/911] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:37.010597 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:37.114390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732237c-48e0-468a-9946-a8fc8fa7c440 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:37.152596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f8899-43b1-41b3-bd66-31fed69b607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:37.153423 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f8899-43b1-41b3-bd66-31fed69b607a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:37.153710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f8899-43b1-41b3-bd66-31fed69b607a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:37.153990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f8899-43b1-41b3-bd66-31fed69b607a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:37.154853 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f8899-43b1-41b3-bd66-31fed69b607a None admin] RBAC: Adding query filter params (role.id=97cdf992a64b4a93b8ecef5c96c7dc35, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=6cf6a2d926e9494abb63a9535b547b6e) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:37.155173 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f8899-43b1-41b3-bd66-31fed69b607a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:37.156641 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f8899-43b1-41b3-bd66-31fed69b607a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:37.163781 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 456/912] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:37 2022] GET /identity/v3/role_assignments?user.id=6cf6a2d926e9494abb63a9535b547b6e&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=97cdf992a64b4a93b8ecef5c96c7dc35 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:37.436078 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542741a-6074-4685-87af-7df81802b50e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:37.436470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542741a-6074-4685-87af-7df81802b50e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:37.436681 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542741a-6074-4685-87af-7df81802b50e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:37.438313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 457/913] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:37.444254 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e55dc-4827-4b16-9068-00b193772a1c None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:37.445179 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e55dc-4827-4b16-9068-00b193772a1c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:37.445681 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e55dc-4827-4b16-9068-00b193772a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:37.463527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-bd7e55dc-4827-4b16-9068-00b193772a1c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:37.465009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-bd7e55dc-4827-4b16-9068-00b193772a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:37.534236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 457/914] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:37.558283 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542741a-6074-4685-87af-7df81802b50e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:37.665189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542741a-6074-4685-87af-7df81802b50e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:37.701022 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:37.701788 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:37.702074 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:37.702356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:37.703019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:37.704390 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:37.716862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:37.717956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:37.718908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 458/915] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 9 15:42:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:37.726985 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e55dc-4827-4b16-9068-00b193772a1c None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:37.838515 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e55dc-4827-4b16-9068-00b193772a1c None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:37.873461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:37.874136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:37.874424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:37.874714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:37.875434 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:37.875747 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:37.877144 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:37.882848 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 458/916] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:37.888480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:38.001590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341470-ffc9-4e5d-9809-fd2619692f4a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:38.037275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55c51c-b220-4718-a428-f2a35cadbb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:38.038327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55c51c-b220-4718-a428-f2a35cadbb2f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.038844 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55c51c-b220-4718-a428-f2a35cadbb2f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.039340 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55c51c-b220-4718-a428-f2a35cadbb2f None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.040278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55c51c-b220-4718-a428-f2a35cadbb2f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:38.040813 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55c51c-b220-4718-a428-f2a35cadbb2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:38.042532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55c51c-b220-4718-a428-f2a35cadbb2f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:38.048333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 459/917] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:38.054268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:38.163481 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9424-f715-4449-aa60-1e4218792102 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:38.198260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b88cb-dc9a-4402-84a2-ef4249b7953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:38.198924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b88cb-dc9a-4402-84a2-ef4249b7953a None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.199215 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b88cb-dc9a-4402-84a2-ef4249b7953a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.199498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b88cb-dc9a-4402-84a2-ef4249b7953a None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.200261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b88cb-dc9a-4402-84a2-ef4249b7953a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:38.201673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b88cb-dc9a-4402-84a2-ef4249b7953a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:38.224738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 459/918] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:42:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:38.304402 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52aee7-047f-417e-8c84-a4ce624f290e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.304601 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52aee7-047f-417e-8c84-a4ce624f290e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.304875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52aee7-047f-417e-8c84-a4ce624f290e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.306714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 460/919] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:38.313262 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6cde5-f073-4163-bd57-2d226ea8c62b None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.313617 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6cde5-f073-4163-bd57-2d226ea8c62b None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.313844 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6cde5-f073-4163-bd57-2d226ea8c62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.333531 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-d3c6cde5-f073-4163-bd57-2d226ea8c62b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:38.334757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-d3c6cde5-f073-4163-bd57-2d226ea8c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:38.420237 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 460/920] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:38.439629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52aee7-047f-417e-8c84-a4ce624f290e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:38.553064 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52aee7-047f-417e-8c84-a4ce624f290e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:38.591012 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b40adf-7a2e-4545-ab7e-61a3b33cc6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:38.592585 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b40adf-7a2e-4545-ab7e-61a3b33cc6aa None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.593130 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b40adf-7a2e-4545-ab7e-61a3b33cc6aa None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.593601 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b40adf-7a2e-4545-ab7e-61a3b33cc6aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.594458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b40adf-7a2e-4545-ab7e-61a3b33cc6aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:38.596042 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b40adf-7a2e-4545-ab7e-61a3b33cc6aa None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-e3b40adf-7a2e-4545-ab7e-61a3b33cc6aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:38.608836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:38.609990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 09 15:42:38.611071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:38.613664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 461/921] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:38.619835 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6cde5-f073-4163-bd57-2d226ea8c62b None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:38.720561 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6cde5-f073-4163-bd57-2d226ea8c62b None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:38.740924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09603bbd-93d2-4769-b026-8c2dc2c9003f None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.741485 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09603bbd-93d2-4769-b026-8c2dc2c9003f None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.741921 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09603bbd-93d2-4769-b026-8c2dc2c9003f None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.743715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 462/922] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:38.749149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a80d6a-7cbd-47ef-9093-9301eee7a12c None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.749715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a80d6a-7cbd-47ef-9093-9301eee7a12c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.750215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a80d6a-7cbd-47ef-9093-9301eee7a12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.756152 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:38.756834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.757158 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.758274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.759020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:38.759336 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:38.760722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:38.766818 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 461/923] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:38.770387 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-76a80d6a-7cbd-47ef-9093-9301eee7a12c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:38.771930 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-76a80d6a-7cbd-47ef-9093-9301eee7a12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:38.773010 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:38.840854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 463/924] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:38.863126 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a80d6a-7cbd-47ef-9093-9301eee7a12c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:38.899234 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e50d-b1e3-4732-9806-807d2c3eb84d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:38.941692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:38.942374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:38.943010 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:38.943316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:38.944049 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:38.944366 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:38.946380 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:38.957123 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 462/925] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 9 15:42:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:38.958969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:38.972484 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a80d6a-7cbd-47ef-9093-9301eee7a12c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:39.010221 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:39.011370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.011823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.012316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.013164 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:39.014812 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:39.027625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:39.028936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:39.029915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:39.037156 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 464/926] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:39.039116 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:39.080406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede8597-dc37-46bb-a950-2e60a43d3e27 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:39.126791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e1478-e3ba-491c-9598-dc72f563f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:39.127688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e1478-e3ba-491c-9598-dc72f563f2e0 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.128615 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e1478-e3ba-491c-9598-dc72f563f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.129629 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e1478-e3ba-491c-9598-dc72f563f2e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.131136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e1478-e3ba-491c-9598-dc72f563f2e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:39.134700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e1478-e3ba-491c-9598-dc72f563f2e0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:39.144860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ae772-b6dc-4b6f-acd3-2848262d8325 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:39.165217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 463/927] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:42:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:42:39.197005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:39.198161 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.198639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.199148 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.200091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:39.200623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:39.202266 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:39.208363 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 465/928] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:39.214704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e1478-e3ba-491c-9598-dc72f563f2e0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:39.336166 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e1478-e3ba-491c-9598-dc72f563f2e0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:39.372516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e400583-28dc-41aa-b40e-7f2d0503c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:39.373242 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400583-28dc-41aa-b40e-7f2d0503c1ed None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.373536 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400583-28dc-41aa-b40e-7f2d0503c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.373816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400583-28dc-41aa-b40e-7f2d0503c1ed None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.374541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e400583-28dc-41aa-b40e-7f2d0503c1ed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:39.374855 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e400583-28dc-41aa-b40e-7f2d0503c1ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:39.376232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e400583-28dc-41aa-b40e-7f2d0503c1ed None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:39.382271 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 464/929] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:39.388033 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:39.479682 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c3d6-4a51-4269-9505-63a004fed7c6 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.480399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c3d6-4a51-4269-9505-63a004fed7c6 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368c3d6-4a51-4269-9505-63a004fed7c6 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.482910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 465/930] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:39.487936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd26cb-a582-4f71-8a5d-64e01f76c3b7 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.488532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd26cb-a582-4f71-8a5d-64e01f76c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.489073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd26cb-a582-4f71-8a5d-64e01f76c3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.497879 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4915d5c-65d4-4ae7-804c-4ad9f99011a5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:39.507204 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-39dd26cb-a582-4f71-8a5d-64e01f76c3b7 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:39.508708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-39dd26cb-a582-4f71-8a5d-64e01f76c3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:39.541937 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:39.542614 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.542902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.543184 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.543993 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] RBAC: Adding query filter params (interface=public, service_id=07098c71c0b64411917866aaeecb1f0f, region_id=RegionOne) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:39.544301 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:39.545797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:39.552858 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 466/931] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:42:39 2022] GET /identity/v3/endpoints?service_id=07098c71c0b64411917866aaeecb1f0f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:39.558514 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:39.587194 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 466/932] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:39.605997 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd26cb-a582-4f71-8a5d-64e01f76c3b7 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:39.666966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580dd98-60ec-478d-bf4e-6dd07e4c2512 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:39.700771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97450867-f9fd-4695-b083-6b58991e220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:39.701749 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450867-f9fd-4695-b083-6b58991e220d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.702252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450867-f9fd-4695-b083-6b58991e220d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.702715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450867-f9fd-4695-b083-6b58991e220d None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.703606 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97450867-f9fd-4695-b083-6b58991e220d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:39.705176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97450867-f9fd-4695-b083-6b58991e220d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:39.708311 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd26cb-a582-4f71-8a5d-64e01f76c3b7 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:39.712504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 467/933] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:42:39 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:42:39.754344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4b476-e991-49b3-ad4f-d7c196f833b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:39.755494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4b476-e991-49b3-ad4f-d7c196f833b1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.755956 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4b476-e991-49b3-ad4f-d7c196f833b1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.756397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4b476-e991-49b3-ad4f-d7c196f833b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.769543 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4b476-e991-49b3-ad4f-d7c196f833b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-48b4b476-e991-49b3-ad4f-d7c196f833b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:39.778524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:39.779903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:39.781076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:39.789163 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 467/934] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:42:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:39.790839 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97450867-f9fd-4695-b083-6b58991e220d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:39.904517 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97450867-f9fd-4695-b083-6b58991e220d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:39.940924 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:39.941653 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:39.941949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:39.942234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:39.943002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:39.943319 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:39.944702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:39.953277 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 468/935] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:42:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:39.959860 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4b476-e991-49b3-ad4f-d7c196f833b1 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:40.064100 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4b476-e991-49b3-ad4f-d7c196f833b1 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:40.100075 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:40.100891 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:40.101220 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:40.101505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:40.112783 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:40.114268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:40.132008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:40.133096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:40.134847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:40.134847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:40.134847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:40.134847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:40.134847 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 468/936] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:40.139155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:40.247572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af46409-6a91-4fd4-8543-bd2ee8ad352e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:40.282199 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:40.282877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:40.283169 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:40.283460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:40.284226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:40.284545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:40.285951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:40.298203 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 469/937] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:40.303756 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:40.419774 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee9096-9c51-4104-8410-a4d0fb4a250c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:40.454265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64c22d-6ab4-4fd7-8837-4bf388bc7b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:40.455102 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64c22d-6ab4-4fd7-8837-4bf388bc7b8b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:40.455393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64c22d-6ab4-4fd7-8837-4bf388bc7b8b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:40.455703 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64c22d-6ab4-4fd7-8837-4bf388bc7b8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:40.463917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64c22d-6ab4-4fd7-8837-4bf388bc7b8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:40.465452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64c22d-6ab4-4fd7-8837-4bf388bc7b8b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-0e64c22d-6ab4-4fd7-8837-4bf388bc7b8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:40.480929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:40.483464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:40.485992 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 469/938] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:42:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:40.490130 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:40.564662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f8800-5386-4416-a9fc-b7a693ea8cc3 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:40.564867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f8800-5386-4416-a9fc-b7a693ea8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:40.565261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f8800-5386-4416-a9fc-b7a693ea8cc3 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:40.566849 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 470/939] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:40.571811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09e44-8c60-4679-b57f-6aebb89b1167 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:40.572101 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09e44-8c60-4679-b57f-6aebb89b1167 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:40.572386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f09e44-8c60-4679-b57f-6aebb89b1167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:40.594997 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-26f09e44-8c60-4679-b57f-6aebb89b1167 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:40.596200 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-26f09e44-8c60-4679-b57f-6aebb89b1167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:40.597832 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6856c-68d5-4cba-93a0-7b5f3c821417 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:40.638023 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:40.639249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:40.639748 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:40.640260 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:40.641523 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:40.642849 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:40.645150 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:40.667673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 470/940] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:42:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:40.673701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:40.681138 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 471/941] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:40.703743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f09e44-8c60-4679-b57f-6aebb89b1167 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:40.776594 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58f15b-ff87-44d5-8462-5d0e84333b27 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:40.801149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f09e44-8c60-4679-b57f-6aebb89b1167 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:40.812001 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:40.813178 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:40.813697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:40.814240 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:40.824989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:40.826750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:40.835385 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:40.836182 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:40.836466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:40.836745 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] PATH_INFO: `/v3/services/compute` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:40.837433 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:40.838782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:40.848034 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:40.849058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:40.850075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:40.850075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:40.850075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:40.850075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:40.851388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:40.852464 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:40.853447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:40.855669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 471/942] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:40.857205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 472/943] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:40.860676 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:40.864444 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:40.972214 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f53aba-2a37-4129-8999-90db7429b833 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:41.002393 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57627c-8257-4ad8-a952-86240726fd67 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:41.007305 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:41.007988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.008324 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.008609 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.009456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:41.009863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:41.011258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:41.028515 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 472/944] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:41.030141 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:41.046615 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:41.047392 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.047595 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.047918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.048686 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] RBAC: Adding query filter params (name=compute) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:41.049002 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:41.050475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:41.060493 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 473/945] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:41.061434 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:41.161426 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabd8c4-9210-4aad-9d85-ee7912cde4ee None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:41.161854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc023d-d972-4245-8bf5-9b51ee7dcd76 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:41.196035 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:41.196712 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.197001 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.197313 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.198034 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] RBAC: Adding query filter params (type=compute) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:41.198347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:41.199922 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:41.213061 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 474/946] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:41.215043 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:41.215884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.216177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.216460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.224844 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:41.226445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:41.242828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:41.243902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:41.244918 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 473/947] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:41.251805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:41.365140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728684-c37a-40d9-9c1a-c93c6caf2238 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:41.402631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faae9d5-e1d8-412d-8155-5c2c7e6cad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:41.403722 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faae9d5-e1d8-412d-8155-5c2c7e6cad36 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.404237 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faae9d5-e1d8-412d-8155-5c2c7e6cad36 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.404726 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faae9d5-e1d8-412d-8155-5c2c7e6cad36 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.405799 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faae9d5-e1d8-412d-8155-5c2c7e6cad36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:41.406334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faae9d5-e1d8-412d-8155-5c2c7e6cad36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:41.408050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faae9d5-e1d8-412d-8155-5c2c7e6cad36 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:41.422490 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 475/948] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:41.429318 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:41.514552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0752-a831-4584-928d-e2f60d9742fa None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.514895 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0752-a831-4584-928d-e2f60d9742fa None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.515296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a0752-a831-4584-928d-e2f60d9742fa None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.516791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 476/949] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:41.522784 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726e55-2b3c-4b0f-84b2-1d68ae139824 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.523069 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726e55-2b3c-4b0f-84b2-1d68ae139824 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.523344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726e55-2b3c-4b0f-84b2-1d68ae139824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.540400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-27726e55-2b3c-4b0f-84b2-1d68ae139824 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:41.542005 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-27726e55-2b3c-4b0f-84b2-1d68ae139824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:41.542863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f3c88-6bc3-4e21-b09c-6df97bf6c378 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:41.587803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550a004-9abc-4c39-91a9-81f35c132a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:41.588698 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550a004-9abc-4c39-91a9-81f35c132a61 None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.588969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550a004-9abc-4c39-91a9-81f35c132a61 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.589282 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550a004-9abc-4c39-91a9-81f35c132a61 None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/6cf6a2d926e9494abb63a9535b547b6e/roles/97cdf992a64b4a93b8ecef5c96c7dc35` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.591976 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550a004-9abc-4c39-91a9-81f35c132a61 None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=6cf6a2d926e9494abb63a9535b547b6e, role_id=97cdf992a64b4a93b8ecef5c96c7dc35)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:41.594296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550a004-9abc-4c39-91a9-81f35c132a61 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:41.611186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 474/950] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:41 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/6cf6a2d926e9494abb63a9535b547b6e/roles/97cdf992a64b4a93b8ecef5c96c7dc35 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:42:41.636483 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 477/951] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:41.661338 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550a004-9abc-4c39-91a9-81f35c132a61 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:41.785169 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550a004-9abc-4c39-91a9-81f35c132a61 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:41.824311 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:41.824988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:41.825306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:41.825585 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:41.826294 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:41.827727 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:41.829307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: WARNING keystone.server.flask.common [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 09 15:42:41.848195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.sql.core [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:42:41.848195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 15:42:41.848195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [parameters: {'id': 'e05dbf18546a402892d8645c8657a3ba', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 15:42:41.848195 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=275711) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e05dbf18546a402892d8645c8657a3ba.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e05dbf18546a402892d8645c8657a3ba. May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application conn.query(q) May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:42:41.855967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result.read() May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 09 15:42:41.856859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return project_ref.to_dict() May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application next(self.gen) May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application yield resource May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application next(self.gen) May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise self.value May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application session.commit() May 09 15:42:41.858006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.session.flush() May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application compat.raise_( May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:42:41.859069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application c = connection._execute_20( May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:42:41.860136 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application conn.query(q) May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result.read() May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:42:41.861213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [parameters: {'id': 'e05dbf18546a402892d8645c8657a3ba', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:41.863373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:41.865836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.Conflict( May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e05dbf18546a402892d8645c8657a3ba. May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:41.870926 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 475/952] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 9 15:42:41 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:41.871885 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27726e55-2b3c-4b0f-84b2-1d68ae139824 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:41.982625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27726e55-2b3c-4b0f-84b2-1d68ae139824 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:42.018779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb33ad5-a6b8-429c-88dd-f5e0bb5b59ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:42.019929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb33ad5-a6b8-429c-88dd-f5e0bb5b59ea None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:42.020424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb33ad5-a6b8-429c-88dd-f5e0bb5b59ea None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:42.020901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb33ad5-a6b8-429c-88dd-f5e0bb5b59ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:42.028886 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb33ad5-a6b8-429c-88dd-f5e0bb5b59ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:42.030758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb33ad5-a6b8-429c-88dd-f5e0bb5b59ea None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-aeb33ad5-a6b8-429c-88dd-f5e0bb5b59ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:42.047519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:42.049590 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:42.051626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:42.051626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:42.051626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:42.051626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:42.051626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:42.051626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:42.051626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:42.051626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:42.054307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 478/953] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:42.061220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:42.109642 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eba84-1d52-466a-8cce-3ae64cd4d641 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:42.110188 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eba84-1d52-466a-8cce-3ae64cd4d641 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:42.110651 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eba84-1d52-466a-8cce-3ae64cd4d641 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:42.112468 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 479/954] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:42.117827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79534b38-7a29-4f90-a5f3-b53c64afa5d4 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:42.118426 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79534b38-7a29-4f90-a5f3-b53c64afa5d4 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:42.118426 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79534b38-7a29-4f90-a5f3-b53c64afa5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:42.135433 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-79534b38-7a29-4f90-a5f3-b53c64afa5d4 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:42.136651 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-79534b38-7a29-4f90-a5f3-b53c64afa5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:42.189931 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14b32a-0821-4cb8-bacc-146c02928666 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:42.217703 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 480/955] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 09 15:42:42.240592 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:42.241615 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:42.242092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:42.242591 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:42.243521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79534b38-7a29-4f90-a5f3-b53c64afa5d4 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:42.243736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:42.244200 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:42.247144 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:42.273711 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 476/956] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:42.363708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79534b38-7a29-4f90-a5f3-b53c64afa5d4 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:42.403061 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:42.403901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:42.404192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:42.404478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:42.405198 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:42.406603 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:42.419917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:42.421353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:42.422534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:42.425078 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 481/957] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:42.428398 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:42.538652 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f84fda-6619-4000-a8eb-cce2b46b974a None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:42.573417 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:42.574378 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:42.574823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:42.575259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:42.576143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:42.576630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:42.578192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:42.584306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 477/958] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:42.593067 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:42.692530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192eec4-05d5-4946-b080-0c70ff21ab79 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:42.726957 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06eef5-3db5-469a-8162-e2cf09ef12d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:42.727630 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06eef5-3db5-469a-8162-e2cf09ef12d8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:42.727923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06eef5-3db5-469a-8162-e2cf09ef12d8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:42.728207 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06eef5-3db5-469a-8162-e2cf09ef12d8 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:42.728929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06eef5-3db5-469a-8162-e2cf09ef12d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:42.729275 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06eef5-3db5-469a-8162-e2cf09ef12d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:42.730660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06eef5-3db5-469a-8162-e2cf09ef12d8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:42.740819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 482/959] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:42.742226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:42.851875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ca7b4-a0fd-4b5a-b223-a7d75c45938c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:42.888385 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce96424-cbb8-4fe5-b25b-00a6d0c8e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:42.889520 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce96424-cbb8-4fe5-b25b-00a6d0c8e91a None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:42.890029 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce96424-cbb8-4fe5-b25b-00a6d0c8e91a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:42.890510 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce96424-cbb8-4fe5-b25b-00a6d0c8e91a None admin] PATH_INFO: `/v3/endpoints` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:42.891480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce96424-cbb8-4fe5-b25b-00a6d0c8e91a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:42.893170 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce96424-cbb8-4fe5-b25b-00a6d0c8e91a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:42.933235 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 478/960] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:42:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:43.558643 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082457d-53c9-42a2-8b67-3857de616875 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:43.559222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082457d-53c9-42a2-8b67-3857de616875 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:43.559222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082457d-53c9-42a2-8b67-3857de616875 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:43.560885 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 483/961] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:43.566773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc5f2-f737-4c57-b461-31271a0362ed None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:43.567082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc5f2-f737-4c57-b461-31271a0362ed None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:43.567376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc5f2-f737-4c57-b461-31271a0362ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:43.585192 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-6bebc5f2-f737-4c57-b461-31271a0362ed None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:43.586742 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-6bebc5f2-f737-4c57-b461-31271a0362ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:43.669210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 479/962] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:43.685645 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082457d-53c9-42a2-8b67-3857de616875 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:43.799556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082457d-53c9-42a2-8b67-3857de616875 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:43.835218 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a12466-6f88-4084-959a-f18a1ac9fdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:43.835823 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12466-6f88-4084-959a-f18a1ac9fdad None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:43.836118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12466-6f88-4084-959a-f18a1ac9fdad None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:43.836463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12466-6f88-4084-959a-f18a1ac9fdad None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:43.837241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a12466-6f88-4084-959a-f18a1ac9fdad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:43.838685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a12466-6f88-4084-959a-f18a1ac9fdad None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:43.861247 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 484/963] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:42:43 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:43.967083 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3794a-db58-4fd1-8f8f-55ac078fc317 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:43.967696 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3794a-db58-4fd1-8f8f-55ac078fc317 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:43.967696 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3794a-db58-4fd1-8f8f-55ac078fc317 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:43.969212 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 480/964] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:43.975386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0047-c4d5-43f7-a248-ac441dcce559 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:43.975686 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0047-c4d5-43f7-a248-ac441dcce559 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:43.976028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0047-c4d5-43f7-a248-ac441dcce559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:43.994137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-6fcc0047-c4d5-43f7-a248-ac441dcce559 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:43.995458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-6fcc0047-c4d5-43f7-a248-ac441dcce559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:44.078256 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 485/965] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:44.103258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3794a-db58-4fd1-8f8f-55ac078fc317 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:44.215308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3794a-db58-4fd1-8f8f-55ac078fc317 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:44.251152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec903-4ec8-4aca-b729-fd34a2f61b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:44.252032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec903-4ec8-4aca-b729-fd34a2f61b54 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:44.252327 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec903-4ec8-4aca-b729-fd34a2f61b54 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:44.253342 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec903-4ec8-4aca-b729-fd34a2f61b54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:44.264242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ec903-4ec8-4aca-b729-fd34a2f61b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-7a3ec903-4ec8-4aca-b729-fd34a2f61b54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:44.272752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:44.273916 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:44.275449 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 481/966] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:42:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:44.286147 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0047-c4d5-43f7-a248-ac441dcce559 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:44.388407 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0047-c4d5-43f7-a248-ac441dcce559 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:44.422397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:44.423085 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:44.423376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:44.423663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:44.424959 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:44.425332 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:44.426713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:44.435390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 486/967] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:42:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:44.441306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec903-4ec8-4aca-b729-fd34a2f61b54 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:44.550278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec903-4ec8-4aca-b729-fd34a2f61b54 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:44.584724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820972d-9244-4c9f-a7db-3483d46e8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:44.585614 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820972d-9244-4c9f-a7db-3483d46e8272 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:44.585911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820972d-9244-4c9f-a7db-3483d46e8272 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:44.586201 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820972d-9244-4c9f-a7db-3483d46e8272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:44.597239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820972d-9244-4c9f-a7db-3483d46e8272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:44.598698 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820972d-9244-4c9f-a7db-3483d46e8272 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-d820972d-9244-4c9f-a7db-3483d46e8272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:44.616889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:44.619210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:44.621571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:44.621571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:44.621571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:44.621571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:44.621571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 482/968] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:44.625059 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:44.655217 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27095f5-5af0-4981-833b-4f3a077d5dbe None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:44.655549 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27095f5-5af0-4981-833b-4f3a077d5dbe None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:44.655809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27095f5-5af0-4981-833b-4f3a077d5dbe None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:44.662272 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 483/969] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:44.663888 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b195-23f0-4ad5-ab74-8a531fab6411 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:44.664345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b195-23f0-4ad5-ab74-8a531fab6411 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:44.664805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b195-23f0-4ad5-ab74-8a531fab6411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:44.683069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-e731b195-23f0-4ad5-ab74-8a531fab6411 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:44.685016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-e731b195-23f0-4ad5-ab74-8a531fab6411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:44.734801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9a6ab-5568-499c-bef5-f14a2a4f4130 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:44.764307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 484/970] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:44.775130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:44.775801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:44.776087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:44.776364 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:44.777128 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:44.777443 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:44.778816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:44.787667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b195-23f0-4ad5-ab74-8a531fab6411 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:44.794372 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 487/971] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:44.800687 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:44.892313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b195-23f0-4ad5-ab74-8a531fab6411 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:44.918174 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ceab1-1469-41de-b639-225cd5a1af5a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:44.936798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:44.937785 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:44.938306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:44.938803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:44.939751 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:44.941497 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:44.954877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.sql.core [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:42:44.954877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 15:42:44.954877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [parameters: {'user_id': 'd7142cbbb25d4e05b452dc1b8723dfec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 15:42:44.954877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=275711) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 09 15:42:44.957946 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:44.958756 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:44.959040 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:44.959318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application conn.query(q) May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result.read() May 09 15:42:44.964638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:42:44.965752 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application next(self.gen) May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application yield resource May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application next(self.gen) May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.session.rollback() May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.force_reraise() May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise self.value May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application session.commit() May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 09 15:42:44.966932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._prepare_impl() May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.session.flush() May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._flush(objects) May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application compat.raise_( May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application flush_context.execute() May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rec.execute(self) May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 09 15:42:44.969137 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application _emit_insert_statements( May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result = connection._execute_20( May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ret = self._execute_context( May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 09 15:42:44.972629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self.dialect.do_execute( May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result = self._query(query) May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application conn.query(q) May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application result.read() May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 09 15:42:44.975004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application packet.raise_for_error() May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd7142cbbb25d4e05b452dc1b8723dfec', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:44.977396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 09 15:42:44.980764 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 09 15:42:44.989058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 09 15:42:44.989058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 09 15:42:44.989058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 09 15:42:44.989058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 09 15:42:44.989058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 09 15:42:44.989058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:44.990269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:44.993913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 09 15:42:44.996382 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:44.999046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 485/973] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 9 15:42:44 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:44.999202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 488/973] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:42:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:44.999202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:44.999511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:45.103773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad1211-5c7e-462a-9ab0-039944554e8b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:45.129072 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2710e14-9a90-4676-ba6f-f1b06862277c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:45.146990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:45.148076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.148580 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.149109 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.150213 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:45.150741 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:45.152383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:45.166533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:45.167345 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.167632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.167925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.172603 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 489/974] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 9 15:42:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:45.178043 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:45.179119 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:45.179468 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:45.198125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:45.199300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:45.200422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 486/975] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:45.205389 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:45.288388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57073aa3-0495-42df-872b-df489c57ce79 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:45.320754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd2528-57b8-4b0a-940b-dfca936d3567 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:45.323205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f15566-9508-469a-ac42-2645b5cdf643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:45.323866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f15566-9508-469a-ac42-2645b5cdf643 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.324155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f15566-9508-469a-ac42-2645b5cdf643 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.324436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f15566-9508-469a-ac42-2645b5cdf643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.356186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecc593-f7e8-4178-b612-16eb64ac74a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:45.356865 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc593-f7e8-4178-b612-16eb64ac74a5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.357347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc593-f7e8-4178-b612-16eb64ac74a5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.357701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecc593-f7e8-4178-b612-16eb64ac74a5 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.359313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc593-f7e8-4178-b612-16eb64ac74a5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:45.359634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc593-f7e8-4178-b612-16eb64ac74a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:45.361303 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecc593-f7e8-4178-b612-16eb64ac74a5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:45.391837 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f15566-9508-469a-ac42-2645b5cdf643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:45.393347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f15566-9508-469a-ac42-2645b5cdf643 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:45.419826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 487/976] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 9 15:42:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:45.419826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2912fe5-6463-4d96-b1c8-6e40b1b06a82 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.419826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2912fe5-6463-4d96-b1c8-6e40b1b06a82 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.419826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2912fe5-6463-4d96-b1c8-6e40b1b06a82 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.419826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 488/977] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:45.420638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc314d-823b-4859-8226-322f0139555a None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.420951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc314d-823b-4859-8226-322f0139555a None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.421270 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc314d-823b-4859-8226-322f0139555a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.444822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-34cc314d-823b-4859-8226-322f0139555a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:45.447138 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-34cc314d-823b-4859-8226-322f0139555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:45.512945 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 490/978] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:45 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:42:45.519399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f15566-9508-469a-ac42-2645b5cdf643 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:45.553244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 489/979] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:45.569787 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc314d-823b-4859-8226-322f0139555a None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:45.645600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f15566-9508-469a-ac42-2645b5cdf643 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:45.690790 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:45.691898 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.692345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.692772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.694496 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc314d-823b-4859-8226-322f0139555a None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:45.703656 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:45.705219 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:45.723791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:45.724954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:45.730390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:45.730390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:45.730390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:45.730390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:45.730390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 491/980] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:45.731893 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:45.739309 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:45.740511 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.740954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.741430 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.742891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:45.745063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:45.763489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:45.764746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:45.766103 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 490/981] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:45.771271 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:45.841159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ac054-eabf-480c-976a-a175b38acb8a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:45.870634 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f814e-778d-4173-9679-8bc74342bb79 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:45.877947 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:45.878736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.879133 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.879777 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.880521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:45.881186 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:45.882843 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:45.902154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 492/982] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:45.905574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:45.908588 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:45.909959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:45.910252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:45.910539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:45.911267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:45.911587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:45.915773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:45.918493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 491/983] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:45.924190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.007136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e6d41-ad5c-4719-bd74-f8de62607b52 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.023417 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624cf99-8a67-4797-86ed-83d81b233407 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.043555 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.044378 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.044669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.044954 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.052530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.054038 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.062504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.063178 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.063464 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.063747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.064463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:46.064778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.066311 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:46.069663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:46.070692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:46.077358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:46.077358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:46.077358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:46.077358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:46.077358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:46.077358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:46.077358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:46.077358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:46.077881 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 492/984] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:46.077975 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 493/985] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:46.078056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.078145 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.180112 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232edd62-1041-4b72-ab21-c73e0910819d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.181272 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc06b-56b5-4f9b-baf6-84454f6ce8a8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.214867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ad4d-f468-471c-8ad0-99e8fb47bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.215088 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3a2fa-a924-40d6-9782-43c210d20c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.215534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ad4d-f468-471c-8ad0-99e8fb47bbf7 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.215718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3a2fa-a924-40d6-9782-43c210d20c0f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.215930 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ad4d-f468-471c-8ad0-99e8fb47bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.216019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3a2fa-a924-40d6-9782-43c210d20c0f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.216115 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ad4d-f468-471c-8ad0-99e8fb47bbf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.216301 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3a2fa-a924-40d6-9782-43c210d20c0f None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.216875 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5ad4d-f468-471c-8ad0-99e8fb47bbf7 None admin] RBAC: Adding query filter params (interface=admin, service_id=07098c71c0b64411917866aaeecb1f0f, region_id=RegionOne) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:46.217093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3a2fa-a924-40d6-9782-43c210d20c0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:46.217202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5ad4d-f468-471c-8ad0-99e8fb47bbf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.217359 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3a2fa-a924-40d6-9782-43c210d20c0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.218581 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5ad4d-f468-471c-8ad0-99e8fb47bbf7 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.218778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3a2fa-a924-40d6-9782-43c210d20c0f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.224414 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 494/986] 10.222.0.13 () {56 vars in 1249 bytes} [Mon May 9 15:42:46 2022] GET /identity/v3/endpoints?service_id=07098c71c0b64411917866aaeecb1f0f&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:46.225810 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34ffa9-9eed-4bce-84e6-86502c146d4e None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.226107 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34ffa9-9eed-4bce-84e6-86502c146d4e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.226391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34ffa9-9eed-4bce-84e6-86502c146d4e None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.227621 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 495/987] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:46.230200 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34ffa9-9eed-4bce-84e6-86502c146d4e None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.231599 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 493/988] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:46.232851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d197-ef5e-46a4-af5b-5909d3b4f72f None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.233239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d197-ef5e-46a4-af5b-5909d3b4f72f None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.233530 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1d197-ef5e-46a4-af5b-5909d3b4f72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.249520 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-27e1d197-ef5e-46a4-af5b-5909d3b4f72f None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:46.250726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-27e1d197-ef5e-46a4-af5b-5909d3b4f72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:46.313771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 494/989] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:46.314527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1d197-ef5e-46a4-af5b-5909d3b4f72f None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.331804 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34ffa9-9eed-4bce-84e6-86502c146d4e None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.364052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58003-c324-4340-9944-6207194cd9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.364710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58003-c324-4340-9944-6207194cd9cb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.365000 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58003-c324-4340-9944-6207194cd9cb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.365313 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58003-c324-4340-9944-6207194cd9cb None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.365996 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd58003-c324-4340-9944-6207194cd9cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.367354 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd58003-c324-4340-9944-6207194cd9cb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.373461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 496/990] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:42:46 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:42:46.375171 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58003-c324-4340-9944-6207194cd9cb None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.418983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1d197-ef5e-46a4-af5b-5909d3b4f72f None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.458751 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.459441 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.459736 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.460022 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.460886 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] RBAC: Adding query filter params (role.id=97cdf992a64b4a93b8ecef5c96c7dc35, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=6cf6a2d926e9494abb63a9535b547b6e) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:46.461236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.462727 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.469651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 495/991] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:46 2022] GET /identity/v3/role_assignments?user.id=6cf6a2d926e9494abb63a9535b547b6e&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=97cdf992a64b4a93b8ecef5c96c7dc35 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:46.485820 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58003-c324-4340-9944-6207194cd9cb None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.526326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.527151 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.527441 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.527721 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] PATH_INFO: `/v3/services/compute` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.528412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.529897 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:46.546518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:46.547845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:46.550124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 497/992] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:46.556620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.664959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602de5c-e7c5-4456-9f89-26c1db73f291 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.700587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.701350 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.701638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.701917 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.702635 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] RBAC: Adding query filter params (name=compute) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:46.702945 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.704313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.709619 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 496/993] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:46.714867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.812311 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd305f61-0971-45a6-ad2d-87b488239a65 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.844564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.845254 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.845544 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.845825 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.846828 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] RBAC: Adding query filter params (type=compute) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:46.847138 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.848496 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:46.853386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 498/994] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:46.859401 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:46.956682 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e2c61-11f2-4db9-9af3-60bb9b468467 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:46.989602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f441c-68c7-4bd5-b2e1-77b7ae124308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:46.990271 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f441c-68c7-4bd5-b2e1-77b7ae124308 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:46.990559 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f441c-68c7-4bd5-b2e1-77b7ae124308 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:46.990837 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f441c-68c7-4bd5-b2e1-77b7ae124308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:46.991627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f441c-68c7-4bd5-b2e1-77b7ae124308 None admin] RBAC: Adding query filter params (interface=public, service_id=5cb9f5b962e1410d98e4713bb54b9141, region_id=RegionOne) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:46.991940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f441c-68c7-4bd5-b2e1-77b7ae124308 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:46.993325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f441c-68c7-4bd5-b2e1-77b7ae124308 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:47.003061 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 497/995] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:42:46 2022] GET /identity/v3/endpoints?service_id=5cb9f5b962e1410d98e4713bb54b9141&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:47.004018 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:47.101136 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f0a6e-a924-48a0-ab69-98cda173b516 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:47.138599 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3bdff-2ddc-480e-bc2e-32efb62b2320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:47.139281 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3bdff-2ddc-480e-bc2e-32efb62b2320 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:47.139573 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3bdff-2ddc-480e-bc2e-32efb62b2320 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:47.139857 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3bdff-2ddc-480e-bc2e-32efb62b2320 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:47.140551 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3bdff-2ddc-480e-bc2e-32efb62b2320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:47.141967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3bdff-2ddc-480e-bc2e-32efb62b2320 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:47.157333 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 499/996] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:42:47 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:42:48.651825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa3003-9c91-43e6-9aec-45f2c33c5403 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:48.652395 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa3003-9c91-43e6-9aec-45f2c33c5403 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:48.652395 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa3003-9c91-43e6-9aec-45f2c33c5403 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:48.653964 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 498/997] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:48.659867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6c519-f9ae-4da7-bf7f-8ff0536f4bcf None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:48.660303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6c519-f9ae-4da7-bf7f-8ff0536f4bcf None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:48.660424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6c519-f9ae-4da7-bf7f-8ff0536f4bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:48.677708 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-68c6c519-f9ae-4da7-bf7f-8ff0536f4bcf None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:48.678928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-68c6c519-f9ae-4da7-bf7f-8ff0536f4bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:48.743922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ed839-d1c4-42e7-9b56-1d59dcdcaf86 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:48.744252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ed839-d1c4-42e7-9b56-1d59dcdcaf86 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:48.744491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ed839-d1c4-42e7-9b56-1d59dcdcaf86 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:48.745094 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 500/998] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:48.746058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 499/999] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:48.750965 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05927b80-d5c1-4324-8c05-434cccccaf98 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:48.751265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05927b80-d5c1-4324-8c05-434cccccaf98 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:48.751550 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05927b80-d5c1-4324-8c05-434cccccaf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:48.767761 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-05927b80-d5c1-4324-8c05-434cccccaf98 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:48.768494 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ed839-d1c4-42e7-9b56-1d59dcdcaf86 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:48.769592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-05927b80-d5c1-4324-8c05-434cccccaf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:48.839617 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 501/1000] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:48.861519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05927b80-d5c1-4324-8c05-434cccccaf98 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:48.879791 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ed839-d1c4-42e7-9b56-1d59dcdcaf86 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:48.916720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:48.918376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:48.918376 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:48.918558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:48.919543 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:48.921247 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:48.934238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:48.935440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:48.936579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 500/1001] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 9 15:42:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:48.941661 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:48.969766 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05927b80-d5c1-4324-8c05-434cccccaf98 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:49.003562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:49.004356 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.004645 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.004926 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] PATH_INFO: `/v3/services/image` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.005620 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:49.006997 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:49.019093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:49.020339 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:49.021780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:49.025312 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 502/1002] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:42:48 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:49.026477 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:49.041411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eac40c-d9e4-46c5-a6f6-60a88024e646 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:49.073852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:49.074533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.074828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.075116 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.075835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:49.076152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:49.077565 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:49.083031 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 501/1003] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:49.088823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:49.138296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd6383-92aa-44c0-b001-efa56639c956 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:49.171354 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:49.171952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.172237 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.172518 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.173256 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] RBAC: Adding query filter params (name=image) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:49.173573 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:49.174973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:49.179797 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 503/1004] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:49.185250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:49.191365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06eee2-e4e2-4532-83a3-aac82a89c0f9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:49.226699 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:49.227539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.227911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.228272 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.229124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:49.229518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:49.231013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:49.236973 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 502/1005] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 9 15:42:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:49.242534 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:49.299536 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae41ac-9802-4e9d-9d8b-f918b63448c3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:49.333669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04aad65-1b4b-4b9b-a353-0f2a9fc86234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:49.334338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aad65-1b4b-4b9b-a353-0f2a9fc86234 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.334624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aad65-1b4b-4b9b-a353-0f2a9fc86234 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.334902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aad65-1b4b-4b9b-a353-0f2a9fc86234 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.335613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04aad65-1b4b-4b9b-a353-0f2a9fc86234 None admin] RBAC: Adding query filter params (type=image) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:49.335925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04aad65-1b4b-4b9b-a353-0f2a9fc86234 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:49.337312 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04aad65-1b4b-4b9b-a353-0f2a9fc86234 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:49.353324 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 504/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:49.353573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8edadb-e1a1-4df3-ada4-c90a1df3d5a1 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:49.386699 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fd295-51a9-4f92-b9ab-f4be52c057a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:49.387384 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fd295-51a9-4f92-b9ab-f4be52c057a0 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.387668 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fd295-51a9-4f92-b9ab-f4be52c057a0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.387947 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fd295-51a9-4f92-b9ab-f4be52c057a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.388712 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fd295-51a9-4f92-b9ab-f4be52c057a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:49.390588 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fd295-51a9-4f92-b9ab-f4be52c057a0 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:49.411022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7999a-7146-4ab4-bc78-2b4ddb202b7c None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.411463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7999a-7146-4ab4-bc78-2b4ddb202b7c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.411854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7999a-7146-4ab4-bc78-2b4ddb202b7c None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.414170 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 505/1007] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:49.416011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 503/1008] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:42:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:42:49.423647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053e1b-fcb0-4821-bdc6-ea12d95f568e None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.424062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053e1b-fcb0-4821-bdc6-ea12d95f568e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.424570 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053e1b-fcb0-4821-bdc6-ea12d95f568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.449666 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-8d053e1b-fcb0-4821-bdc6-ea12d95f568e None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:49.451658 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-8d053e1b-fcb0-4821-bdc6-ea12d95f568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:49.561421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 506/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:49.585897 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fd295-51a9-4f92-b9ab-f4be52c057a0 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:49.703489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fd295-51a9-4f92-b9ab-f4be52c057a0 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:49.739244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:49.740065 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.740352 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.740628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.741322 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:49.742700 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:49.756315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:49.757602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:49.758608 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 504/1010] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:49.763892 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d053e1b-fcb0-4821-bdc6-ea12d95f568e None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:49.867981 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d053e1b-fcb0-4821-bdc6-ea12d95f568e None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:49.908308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:49.909738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:49.910141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:49.910523 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:49.911379 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] RBAC: Adding query filter params (name=compute) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:49.911794 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:49.915418 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:49.926342 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 507/1011] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:49.927637 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:50.036652 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ade88-11cd-4872-8afd-b2ed5e6b9ef0 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:50.070110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b2301-073b-4572-8ed3-085826bd0a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:50.070793 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b2301-073b-4572-8ed3-085826bd0a9e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:50.071080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b2301-073b-4572-8ed3-085826bd0a9e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:50.071360 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b2301-073b-4572-8ed3-085826bd0a9e None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:50.072077 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b2301-073b-4572-8ed3-085826bd0a9e None admin] RBAC: Adding query filter params (type=compute) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:50.072389 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b2301-073b-4572-8ed3-085826bd0a9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:50.074132 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b2301-073b-4572-8ed3-085826bd0a9e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:50.081222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 505/1012] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 9 15:42:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:50.086655 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:50.182468 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016b874-954b-4612-8b09-d8cccceb4e18 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:50.214455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3e02b-42db-416b-b7fb-1dbd0335cb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:50.215119 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3e02b-42db-416b-b7fb-1dbd0335cb1a None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:50.215404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3e02b-42db-416b-b7fb-1dbd0335cb1a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:50.215683 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3e02b-42db-416b-b7fb-1dbd0335cb1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:50.216441 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3e02b-42db-416b-b7fb-1dbd0335cb1a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:50.217833 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3e02b-42db-416b-b7fb-1dbd0335cb1a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:50.245072 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 508/1013] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:42:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:42:51.649919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43730d1-e14a-4828-b356-9fcd52f47249 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:51.650491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43730d1-e14a-4828-b356-9fcd52f47249 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:51.650491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43730d1-e14a-4828-b356-9fcd52f47249 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:51.651891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 506/1014] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:51.657246 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29705-0ac0-45d3-b994-676ad1516a6c None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:51.657663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29705-0ac0-45d3-b994-676ad1516a6c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:51.657854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29705-0ac0-45d3-b994-676ad1516a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:51.675640 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-a7d29705-0ac0-45d3-b994-676ad1516a6c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:51.676948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-a7d29705-0ac0-45d3-b994-676ad1516a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:51.717328 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106371f3-2de4-49c8-ae7e-d01308ad1445 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:51.717669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106371f3-2de4-49c8-ae7e-d01308ad1445 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:51.717911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106371f3-2de4-49c8-ae7e-d01308ad1445 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:51.719422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 507/1015] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:51.725002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e318c78-bd76-4b2e-8810-2e960cd6516b None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:51.725336 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e318c78-bd76-4b2e-8810-2e960cd6516b None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:51.725625 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e318c78-bd76-4b2e-8810-2e960cd6516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:51.742342 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-3e318c78-bd76-4b2e-8810-2e960cd6516b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:51.743545 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-3e318c78-bd76-4b2e-8810-2e960cd6516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:51.749842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 509/1016] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:51 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:51.772690 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d29705-0ac0-45d3-b994-676ad1516a6c None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:51.809227 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 508/1017] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:51 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:51.832793 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e318c78-bd76-4b2e-8810-2e960cd6516b None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:51.868535 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d29705-0ac0-45d3-b994-676ad1516a6c None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:51.902032 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e3a5a-f219-4ad3-8e3c-a63f9642e50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:51.905556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e3a5a-f219-4ad3-8e3c-a63f9642e50e None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:51.905556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e3a5a-f219-4ad3-8e3c-a63f9642e50e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:51.905556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e3a5a-f219-4ad3-8e3c-a63f9642e50e None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:51.905556 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e3a5a-f219-4ad3-8e3c-a63f9642e50e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:51.906042 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e3a5a-f219-4ad3-8e3c-a63f9642e50e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:51.929282 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 510/1018] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:42:51 2022] POST /identity/v3/services => generated 242 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 09 15:42:51.932544 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e318c78-bd76-4b2e-8810-2e960cd6516b None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:51.977050 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0239774-e9a6-4b4d-b60b-24e2bc83f335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:51.977922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0239774-e9a6-4b4d-b60b-24e2bc83f335 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:51.978215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0239774-e9a6-4b4d-b60b-24e2bc83f335 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:51.978499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0239774-e9a6-4b4d-b60b-24e2bc83f335 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:51.979779 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0239774-e9a6-4b4d-b60b-24e2bc83f335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:51.981759 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0239774-e9a6-4b4d-b60b-24e2bc83f335 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:51.988688 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 509/1019] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:51.990031 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e3a5a-f219-4ad3-8e3c-a63f9642e50e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:52.100917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e3a5a-f219-4ad3-8e3c-a63f9642e50e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:52.139315 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364362d1-3e11-4bde-bbe8-0bdcfb78954f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:52.140052 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364362d1-3e11-4bde-bbe8-0bdcfb78954f None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:52.140344 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364362d1-3e11-4bde-bbe8-0bdcfb78954f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:52.140632 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364362d1-3e11-4bde-bbe8-0bdcfb78954f None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:52.141388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364362d1-3e11-4bde-bbe8-0bdcfb78954f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:52.143427 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364362d1-3e11-4bde-bbe8-0bdcfb78954f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:52.169013 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 511/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:42:51 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:52.596979 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b931eb-adf1-470c-bf44-064c30b2bec0 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:52.597381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b931eb-adf1-470c-bf44-064c30b2bec0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:52.597480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b931eb-adf1-470c-bf44-064c30b2bec0 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:52.599003 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 510/1021] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:52.604159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344ef97-d50c-4aed-9a00-455ad5d5b761 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:52.604457 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344ef97-d50c-4aed-9a00-455ad5d5b761 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:52.604743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344ef97-d50c-4aed-9a00-455ad5d5b761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:52.621551 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-d344ef97-d50c-4aed-9a00-455ad5d5b761 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:52.622742 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-d344ef97-d50c-4aed-9a00-455ad5d5b761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:52.687800 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 512/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:52.711669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b931eb-adf1-470c-bf44-064c30b2bec0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:52.811468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b931eb-adf1-470c-bf44-064c30b2bec0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:52.844983 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39069e-e4bd-4950-818a-f043488e1d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:52.846120 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39069e-e4bd-4950-818a-f043488e1d86 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:52.846885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39069e-e4bd-4950-818a-f043488e1d86 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:52.847428 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39069e-e4bd-4950-818a-f043488e1d86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:52.857566 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39069e-e4bd-4950-818a-f043488e1d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-db39069e-e4bd-4950-818a-f043488e1d86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:52.866616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:52.867723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:52.868740 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:52.870528 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 511/1023] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:42:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:52.876406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344ef97-d50c-4aed-9a00-455ad5d5b761 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:52.974330 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344ef97-d50c-4aed-9a00-455ad5d5b761 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:53.006754 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:53.007424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:53.007705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:53.007980 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:53.008732 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:53.009062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:53.010473 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:53.018534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 513/1024] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:42:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:53.024071 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39069e-e4bd-4950-818a-f043488e1d86 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:53.122525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39069e-e4bd-4950-818a-f043488e1d86 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:53.154259 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:53.155058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:53.155341 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:53.155624 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:53.166440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:53.167888 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:53.184948 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:53.186032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:53.187105 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:53.187105 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:53.187105 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:53.187105 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:53.189357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 512/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:53.191986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:53.293952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de88bfa-b4ad-4fd7-bb5b-c2b858b7ebca None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:53.325279 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:53.325942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:53.326225 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:53.326501 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:53.327218 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:53.327528 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:53.328885 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:53.345085 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 514/1026] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:53.348655 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:53.445253 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3702a-95ef-4397-8b41-98cc011acab5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:53.477357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:53.478170 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:53.478460 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:53.478744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] PATH_INFO: `/v3/users/nova` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:53.486325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:53.488343 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:53.503296 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:53.504462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:53.506152 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:53.511207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 513/1027] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:42:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:53.511334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:53.606505 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b0a7-d879-42a7-a2df-510c5693f92b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:53.639852 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:53.640530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:53.640819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:53.641130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:53.641979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:53.642296 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:53.643748 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:53.665600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 515/1028] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:42:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:53.667443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:53.764387 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f6d5b-26e1-4a10-82d8-ec64bbf7d947 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:53.796773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445e2fd-f8e7-489a-a49f-72032edb133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:53.797465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e2fd-f8e7-489a-a49f-72032edb133b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:53.797756 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e2fd-f8e7-489a-a49f-72032edb133b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:53.798040 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445e2fd-f8e7-489a-a49f-72032edb133b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:53.863522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445e2fd-f8e7-489a-a49f-72032edb133b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:53.864977 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445e2fd-f8e7-489a-a49f-72032edb133b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:53.964652 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 514/1029] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:53 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:42:53.966562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.069350 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33ea3c-ba95-48e1-ad28-4c33028a860e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.104280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c548bd0-452f-4941-a749-5f57b61f12ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.105103 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c548bd0-452f-4941-a749-5f57b61f12ba None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.105397 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c548bd0-452f-4941-a749-5f57b61f12ba None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.105680 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c548bd0-452f-4941-a749-5f57b61f12ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.115735 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c548bd0-452f-4941-a749-5f57b61f12ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.117218 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c548bd0-452f-4941-a749-5f57b61f12ba None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-4c548bd0-452f-4941-a749-5f57b61f12ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:54.134819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:54.135918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:54.141624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:54.141624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:54.141624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:54.141624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:54.141624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 516/1030] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:54.144323 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445e2fd-f8e7-489a-a49f-72032edb133b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.198647 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b2cc9-ae7d-4984-b02e-f905e0d9cfec None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.199006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b2cc9-ae7d-4984-b02e-f905e0d9cfec None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.199221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b2cc9-ae7d-4984-b02e-f905e0d9cfec None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.200773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 517/1031] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:54.206050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62da94-902a-4d5c-b849-3d4e9560a9cd None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.206348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62da94-902a-4d5c-b849-3d4e9560a9cd None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.206636 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62da94-902a-4d5c-b849-3d4e9560a9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.223945 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-bb62da94-902a-4d5c-b849-3d4e9560a9cd None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:54.225186 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-bb62da94-902a-4d5c-b849-3d4e9560a9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:54.243033 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445e2fd-f8e7-489a-a49f-72032edb133b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.277851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.278558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.278853 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.279143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.280118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:54.280261 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.281803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.299816 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 515/1032] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:54.300034 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 518/1033] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:54.301649 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.317812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62da94-902a-4d5c-b849-3d4e9560a9cd None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.402939 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a4f33-aea3-4dc3-ba2e-4934bcfb59a4 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.415662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62da94-902a-4d5c-b849-3d4e9560a9cd None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.435394 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa11f1-d17f-4708-9167-fb65ea4c404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.436202 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa11f1-d17f-4708-9167-fb65ea4c404a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.436496 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa11f1-d17f-4708-9167-fb65ea4c404a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.436778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa11f1-d17f-4708-9167-fb65ea4c404a None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.444253 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa11f1-d17f-4708-9167-fb65ea4c404a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.446058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa11f1-d17f-4708-9167-fb65ea4c404a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.453030 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.453853 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.454130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.454415 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] PATH_INFO: `/v3/services/image` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.455079 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.456999 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-60fa11f1-d17f-4708-9167-fb65ea4c404a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:54.463186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:54.464242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:54.465370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:54.465370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:54.465370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:54.465370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:54.465370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:54.465370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:54.465370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:54.465370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:54.472521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:54.473871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:54.475465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:54.476359 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 516/1034] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:54.476443 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 519/1035] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:54.476522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acef4f-0245-4faf-979a-822656b96963 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.476522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acef4f-0245-4faf-979a-822656b96963 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.476522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acef4f-0245-4faf-979a-822656b96963 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.476522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 517/1036] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:54.476813 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.477275 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acef4f-0245-4faf-979a-822656b96963 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.574016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acef4f-0245-4faf-979a-822656b96963 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.574855 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b666e-9b9b-452f-8ce2-2b112512148b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.607833 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.608025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480fecf-8a62-4868-8462-06d3912b41aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.608493 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.608600 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480fecf-8a62-4868-8462-06d3912b41aa None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.608780 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.608908 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480fecf-8a62-4868-8462-06d3912b41aa None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.609083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.609189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480fecf-8a62-4868-8462-06d3912b41aa None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.609805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480fecf-8a62-4868-8462-06d3912b41aa None admin] RBAC: Adding query filter params (name=image) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:54.609903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:54.610123 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480fecf-8a62-4868-8462-06d3912b41aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.610224 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.611532 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480fecf-8a62-4868-8462-06d3912b41aa None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.611699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.616254 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 518/1037] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:54.617467 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f54e0f-f8be-4aaf-aade-4fe89900b576 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.617763 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f54e0f-f8be-4aaf-aade-4fe89900b576 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.618050 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f54e0f-f8be-4aaf-aade-4fe89900b576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.629335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 520/1038] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:54.629335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.635621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-b4f54e0f-f8be-4aaf-aade-4fe89900b576 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:54.636826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-b4f54e0f-f8be-4aaf-aade-4fe89900b576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:54.700737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 519/1039] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:54.701965 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f54e0f-f8be-4aaf-aade-4fe89900b576 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.720401 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b7bea-c206-4bee-afb4-ce056a667084 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.754447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.755119 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.755405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.755685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.756398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] RBAC: Adding query filter params (type=image) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:54.756718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.758121 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.762755 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 521/1040] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:54.763529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.795797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f54e0f-f8be-4aaf-aade-4fe89900b576 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.829228 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.829891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.830183 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.830829 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.831303 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] RBAC: Adding query filter params (role.id=97cdf992a64b4a93b8ecef5c96c7dc35, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=ef8546c698f9455cb81264bcee0d2344) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:54.831623 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.833075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.838394 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 520/1041] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/role_assignments?user.id=ef8546c698f9455cb81264bcee0d2344&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=97cdf992a64b4a93b8ecef5c96c7dc35 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:54.839185 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.859226 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350292f3-bb32-433d-bfcd-b90e9d0b2ecf None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.896525 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bd29e-f297-4854-842e-7f74c184a42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.897375 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd29e-f297-4854-842e-7f74c184a42b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.897668 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd29e-f297-4854-842e-7f74c184a42b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.897952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bd29e-f297-4854-842e-7f74c184a42b None admin] PATH_INFO: `/v3/domains/default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.899273 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bd29e-f297-4854-842e-7f74c184a42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.900823 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bd29e-f297-4854-842e-7f74c184a42b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.907791 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 522/1042] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:54.908811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bd29e-f297-4854-842e-7f74c184a42b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:54.947154 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8f84-eaf7-4afe-aa70-0094f12b056b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:54.981710 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:54.982304 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:54.982599 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:54.982883 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:54.983693 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] RBAC: Adding query filter params (interface=public, service_id=d29d0a4b61e042d4bab491309b3b4770, region_id=RegionOne) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:54.984012 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:54.985443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:54.992663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 521/1043] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/endpoints?service_id=d29d0a4b61e042d4bab491309b3b4770&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:54.997443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:55.016624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bd29e-f297-4854-842e-7f74c184a42b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:55.052536 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880452fa-ded9-4ab6-a001-42e7713833b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:55.053321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880452fa-ded9-4ab6-a001-42e7713833b3 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:55.053605 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880452fa-ded9-4ab6-a001-42e7713833b3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:55.053886 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880452fa-ded9-4ab6-a001-42e7713833b3 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:55.054628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880452fa-ded9-4ab6-a001-42e7713833b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:55.056062 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880452fa-ded9-4ab6-a001-42e7713833b3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:55.076602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 523/1044] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:42:54 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:55.106242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda3f70-f471-4b5a-bd10-bd013679e0b8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:55.143551 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cb782-e070-4c17-b104-fda99a70d0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:55.144256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb782-e070-4c17-b104-fda99a70d0e0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:55.144547 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb782-e070-4c17-b104-fda99a70d0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:55.144832 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cb782-e070-4c17-b104-fda99a70d0e0 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:55.145581 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cb782-e070-4c17-b104-fda99a70d0e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:55.147004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cb782-e070-4c17-b104-fda99a70d0e0 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:55.161096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 522/1045] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:42:54 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 09 15:42:57.066767 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d588ee-6fdf-4dde-ae64-8d9526569511 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.067360 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d588ee-6fdf-4dde-ae64-8d9526569511 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.067360 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d588ee-6fdf-4dde-ae64-8d9526569511 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.068740 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 524/1046] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:57.074183 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad529c0a-a404-434f-b0f8-7c4a2c0df868 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.074621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad529c0a-a404-434f-b0f8-7c4a2c0df868 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.074822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad529c0a-a404-434f-b0f8-7c4a2c0df868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.091616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-ad529c0a-a404-434f-b0f8-7c4a2c0df868 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:57.092794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-ad529c0a-a404-434f-b0f8-7c4a2c0df868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:57.161860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 523/1047] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:57.184410 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d588ee-6fdf-4dde-ae64-8d9526569511 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:57.295437 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d588ee-6fdf-4dde-ae64-8d9526569511 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:57.328263 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fad93-8e6b-47e0-b847-badb2670d9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:57.329105 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fad93-8e6b-47e0-b847-badb2670d9eb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.329394 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fad93-8e6b-47e0-b847-badb2670d9eb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.329672 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fad93-8e6b-47e0-b847-badb2670d9eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.339915 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fad93-8e6b-47e0-b847-badb2670d9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-907fad93-8e6b-47e0-b847-badb2670d9eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:57.348387 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:57.354889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:57.355993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 525/1048] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:57.356890 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad529c0a-a404-434f-b0f8-7c4a2c0df868 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:57.423637 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb3104-d080-4426-bbd5-885986b359f7 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.424661 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb3104-d080-4426-bbd5-885986b359f7 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.424802 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb3104-d080-4426-bbd5-885986b359f7 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.426179 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 526/1049] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:42:57.431706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166ec21-9ebb-45c9-a1b5-2874c697a2ba None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.431877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166ec21-9ebb-45c9-a1b5-2874c697a2ba None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.432082 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166ec21-9ebb-45c9-a1b5-2874c697a2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.448534 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-8166ec21-9ebb-45c9-a1b5-2874c697a2ba None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:57.449714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-8166ec21-9ebb-45c9-a1b5-2874c697a2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:57.458457 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad529c0a-a404-434f-b0f8-7c4a2c0df868 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:57.491955 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12293c-a908-4853-95f6-b6474b7f47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:57.492632 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12293c-a908-4853-95f6-b6474b7f47e3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.492922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12293c-a908-4853-95f6-b6474b7f47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.493243 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12293c-a908-4853-95f6-b6474b7f47e3 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.494000 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12293c-a908-4853-95f6-b6474b7f47e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:57.494316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12293c-a908-4853-95f6-b6474b7f47e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:57.495678 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12293c-a908-4853-95f6-b6474b7f47e3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:57.503459 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 524/1050] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:57.504750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ce1f5-5a4a-41d2-96d8-062224836e7e None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.505074 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ce1f5-5a4a-41d2-96d8-062224836e7e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.505364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ce1f5-5a4a-41d2-96d8-062224836e7e None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.506603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 525/1051] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:42:57.509075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ce1f5-5a4a-41d2-96d8-062224836e7e None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:57.519074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 527/1052] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:42:57.520758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1e51-3198-4f44-841e-d40d092dc72c None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.521139 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1e51-3198-4f44-841e-d40d092dc72c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.521481 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1e51-3198-4f44-841e-d40d092dc72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.539402 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-5e3f1e51-3198-4f44-841e-d40d092dc72c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:42:57.540628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-5e3f1e51-3198-4f44-841e-d40d092dc72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:42:57.604589 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 528/1053] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:42:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:42:57.605465 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1e51-3198-4f44-841e-d40d092dc72c None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:57.610207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ce1f5-5a4a-41d2-96d8-062224836e7e None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:57.642306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:57.643106 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.643394 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.643675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.653761 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:57.655232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:57.672113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:57.673373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:57.675325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:57.675325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:57.675325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:57.675325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:57.675325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 526/1054] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:57.675325 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:57.704559 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1e51-3198-4f44-841e-d40d092dc72c None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:57.737851 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:57.738645 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.738930 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.739210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] PATH_INFO: `/v3/services/image` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.739867 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:57.741402 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:57.758479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:57.760009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:42:57.761205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 529/1055] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:57.762222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:57.772524 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f7dce-738a-4123-8260-a44c5db25479 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:57.809069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb085a-8328-46b1-aacb-9a2a8120513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:57.809885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb085a-8328-46b1-aacb-9a2a8120513d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.810172 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb085a-8328-46b1-aacb-9a2a8120513d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.810455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb085a-8328-46b1-aacb-9a2a8120513d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.820754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb085a-8328-46b1-aacb-9a2a8120513d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-98fb085a-8328-46b1-aacb-9a2a8120513d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:42:57.832866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:57.834347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 527/1056] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:57.835365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb085a-8328-46b1-aacb-9a2a8120513d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:57.874799 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b7773-36ed-4794-8280-ec96833086d5 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:57.907185 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:57.907853 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.908144 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.908995 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.909821 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:57.910137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:57.911516 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:57.926073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 530/1057] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:57.926073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:57.933974 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb085a-8328-46b1-aacb-9a2a8120513d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:57.966398 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:57.967060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:57.967347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:57.967628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:57.968338 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] RBAC: Adding query filter params (name=image) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:57.968647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:57.970210 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:57.975727 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 528/1058] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:57.976503 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.022500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13643b3-8b88-4721-b679-6958d8e7db6c None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.054815 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.055522 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.055812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.056093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.056846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:58.057193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.058560 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.066467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 531/1059] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:58.067363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.070474 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359773e6-e710-4af9-85e4-a12ea19defae None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.104163 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.104963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.105285 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.105572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] PATH_INFO: `/v3/users/nova` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.113166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.114659 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:42:58.132835 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:58.134302 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:58.135458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 529/1060] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:42:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:58.136702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.172968 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b55d8-2052-48d8-a77e-ef7ab036330b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.205824 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.206491 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.206784 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.207064 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.207776 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] RBAC: Adding query filter params (type=image) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:58.208087 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.209478 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.214193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 532/1061] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:58.214981 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.235266 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ede29-61c5-4679-b1e3-06fde7b12252 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.267035 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a6699-a037-4b46-8924-5b2823594b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.268586 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6699-a037-4b46-8924-5b2823594b8c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.268882 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6699-a037-4b46-8924-5b2823594b8c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.269329 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6699-a037-4b46-8924-5b2823594b8c None admin] PATH_INFO: `/v3/users/45533fa3735c40918eb22e2592fc23aa` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.282104 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a6699-a037-4b46-8924-5b2823594b8c None admin] RBAC: Authorizing `identity:get_user(user_id=45533fa3735c40918eb22e2592fc23aa)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.283562 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a6699-a037-4b46-8924-5b2823594b8c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.285244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 530/1062] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/users/45533fa3735c40918eb22e2592fc23aa => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:58.285988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a6699-a037-4b46-8924-5b2823594b8c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.310154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec7605-5312-48c5-8e98-3036c068b625 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.342524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.343193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.343482 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.343761 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.344604 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:58.344924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.346706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.367792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 533/1063] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:42:58.367792 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.380134 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a6699-a037-4b46-8924-5b2823594b8c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.412331 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ddca9-4257-4842-92ba-f97ac462a216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.412986 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ddca9-4257-4842-92ba-f97ac462a216 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.413300 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ddca9-4257-4842-92ba-f97ac462a216 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.413582 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ddca9-4257-4842-92ba-f97ac462a216 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.414320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ddca9-4257-4842-92ba-f97ac462a216 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.415685 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ddca9-4257-4842-92ba-f97ac462a216 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.433395 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 531/1064] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:42:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:42:58.437981 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ddca9-4257-4842-92ba-f97ac462a216 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.462229 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0698d-a926-4b44-9191-3106e2c0d435 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.507193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268be8e-a1c0-4cfc-8a93-f1df9dafa2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.507772 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be8e-a1c0-4cfc-8a93-f1df9dafa2b4 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.508048 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be8e-a1c0-4cfc-8a93-f1df9dafa2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.508331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be8e-a1c0-4cfc-8a93-f1df9dafa2b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.544726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ddca9-4257-4842-92ba-f97ac462a216 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.585704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.586103 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268be8e-a1c0-4cfc-8a93-f1df9dafa2b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.587558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.587558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.587558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.588154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268be8e-a1c0-4cfc-8a93-f1df9dafa2b4 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.600476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.602030 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:42:58.620158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:58.621372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:58.622456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:42:58.622456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:42:58.622456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:42:58.622456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:58.622456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 532/1065] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:42:58.628767 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.689639 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 534/1066] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:42:58.695491 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268be8e-a1c0-4cfc-8a93-f1df9dafa2b4 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.729957 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40d127-1e11-4877-a414-b49e76d69ec3 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.762689 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.763283 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.763571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.763847 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.764576 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:58.764887 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.766362 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.778622 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 533/1067] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:58.784704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.795320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268be8e-a1c0-4cfc-8a93-f1df9dafa2b4 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.829351 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cb97c-24a2-4cbc-a0ea-d9920a92bcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.830754 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cb97c-24a2-4cbc-a0ea-d9920a92bcd3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.831044 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cb97c-24a2-4cbc-a0ea-d9920a92bcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.831328 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cb97c-24a2-4cbc-a0ea-d9920a92bcd3 None admin] PATH_INFO: `/v3/projects/3800f7bfd9e2467c873821f2ef596d98` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.832605 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cb97c-24a2-4cbc-a0ea-d9920a92bcd3 None admin] RBAC: Authorizing `identity:get_project(project_id=3800f7bfd9e2467c873821f2ef596d98)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.834089 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cb97c-24a2-4cbc-a0ea-d9920a92bcd3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.835966 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 535/1068] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/projects/3800f7bfd9e2467c873821f2ef596d98 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:58.841724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cb97c-24a2-4cbc-a0ea-d9920a92bcd3 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.886232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5044ddb-c828-42b8-aaf3-3fb097d85db9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.925063 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.925878 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.926173 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.926462 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.933202 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.934660 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:58.943840 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cb97c-24a2-4cbc-a0ea-d9920a92bcd3 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:42:58.948806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:42:58.950079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:42:58.951159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 534/1069] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:42:58.956315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:58.982452 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:58.983119 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:58.983412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:58.983699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:58.984552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=3800f7bfd9e2467c873821f2ef596d98, user.id=45533fa3735c40918eb22e2592fc23aa) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:58.984874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:58.986376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:59.001071 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 536/1070] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/role_assignments?user.id=45533fa3735c40918eb22e2592fc23aa&scope.project.id=3800f7bfd9e2467c873821f2ef596d98&role.id=ebff922955b74c919d58613875326151 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:59.059631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835585-9a26-4138-8790-1482fb8a7401 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:59.095743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868071b-1f47-4e5f-b339-de5516d225ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:59.096692 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868071b-1f47-4e5f-b339-de5516d225ac None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:59.096982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868071b-1f47-4e5f-b339-de5516d225ac None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:59.097294 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868071b-1f47-4e5f-b339-de5516d225ac None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:59.098110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868071b-1f47-4e5f-b339-de5516d225ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:42:59.098427 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868071b-1f47-4e5f-b339-de5516d225ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:59.099895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868071b-1f47-4e5f-b339-de5516d225ac None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:59.118642 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 535/1071] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:42:59.120193 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:42:59.223916 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636c261-d5c0-4876-8d10-4afaeaf71ce6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:42:59.259412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cd89a-bc1b-465b-bea3-dd5e965995c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:42:59.260563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd89a-bc1b-465b-bea3-dd5e965995c0 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:42:59.261064 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd89a-bc1b-465b-bea3-dd5e965995c0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:42:59.261507 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cd89a-bc1b-465b-bea3-dd5e965995c0 None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/ef8546c698f9455cb81264bcee0d2344/roles/97cdf992a64b4a93b8ecef5c96c7dc35` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:42:59.263925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cd89a-bc1b-465b-bea3-dd5e965995c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=ef8546c698f9455cb81264bcee0d2344, role_id=97cdf992a64b4a93b8ecef5c96c7dc35)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:42:59.265596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cd89a-bc1b-465b-bea3-dd5e965995c0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:42:59.284743 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 537/1072] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:42:59 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/ef8546c698f9455cb81264bcee0d2344/roles/97cdf992a64b4a93b8ecef5c96c7dc35 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:43:00.696113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024374-ad52-47f3-8003-e7a27aeff349 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:00.696694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024374-ad52-47f3-8003-e7a27aeff349 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:00.696694 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d024374-ad52-47f3-8003-e7a27aeff349 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:00.698220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 536/1073] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:00.703357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed54ff-91d6-4010-99cf-21f7a8569793 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:00.703765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed54ff-91d6-4010-99cf-21f7a8569793 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:00.703946 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed54ff-91d6-4010-99cf-21f7a8569793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:00.720911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-40ed54ff-91d6-4010-99cf-21f7a8569793 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:00.722391 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-40ed54ff-91d6-4010-99cf-21f7a8569793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:00.786483 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 538/1074] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:00.808556 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d024374-ad52-47f3-8003-e7a27aeff349 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:00.907639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d024374-ad52-47f3-8003-e7a27aeff349 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:00.941717 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1eade-a0b8-4b5a-b323-6885362197c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:00.942373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1eade-a0b8-4b5a-b323-6885362197c5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:00.942658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1eade-a0b8-4b5a-b323-6885362197c5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:00.942938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1eade-a0b8-4b5a-b323-6885362197c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.008056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1eade-a0b8-4b5a-b323-6885362197c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:01.009446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1eade-a0b8-4b5a-b323-6885362197c5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:01.109568 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 537/1075] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:43:01.115579 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed54ff-91d6-4010-99cf-21f7a8569793 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.216166 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed54ff-91d6-4010-99cf-21f7a8569793 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:01.222313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3b9b-f45d-4f67-b42b-54d987bd4386 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.222621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3b9b-f45d-4f67-b42b-54d987bd4386 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.222897 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3b9b-f45d-4f67-b42b-54d987bd4386 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.224434 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 538/1076] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:01.229347 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58694-364d-44c0-a31d-171264d2338c None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.229641 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58694-364d-44c0-a31d-171264d2338c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.229917 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a58694-364d-44c0-a31d-171264d2338c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.246456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-12a58694-364d-44c0-a31d-171264d2338c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:01.247660 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-12a58694-364d-44c0-a31d-171264d2338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:01.250834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:01.251628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.252454 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.252735 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.262899 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:01.264279 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:43:01.283154 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:01.284253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:01.290422 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:43:01.290422 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:43:01.290422 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:43:01.290422 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:43:01.290422 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 539/1077] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:01.292022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.317078 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 539/1078] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:43:01.340369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58694-364d-44c0-a31d-171264d2338c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.396684 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb2bd-0f7b-4d4f-85e2-533b600a6716 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:01.430361 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:01.431029 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.431316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.431596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.432326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:01.432625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:01.434051 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:01.443251 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a58694-364d-44c0-a31d-171264d2338c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:01.457070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 540/1079] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:43:01.481177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b7637-5050-42bf-9762-2c8e0ed8d711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:01.482016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7637-5050-42bf-9762-2c8e0ed8d711 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.482306 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7637-5050-42bf-9762-2c8e0ed8d711 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.482588 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7637-5050-42bf-9762-2c8e0ed8d711 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.495551 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b7637-5050-42bf-9762-2c8e0ed8d711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-966b7637-5050-42bf-9762-2c8e0ed8d711 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:01.504367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:01.505598 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:01.506618 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:01.508911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 540/1080] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:43:01.512235 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.533666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d24e-4a20-46f0-8156-6345d7689f66 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.533892 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d24e-4a20-46f0-8156-6345d7689f66 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.534141 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d24e-4a20-46f0-8156-6345d7689f66 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.535806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 541/1081] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:01.541463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffaa17-73ae-4055-a87f-45fb8e3b3bac None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.541780 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffaa17-73ae-4055-a87f-45fb8e3b3bac None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.542058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffaa17-73ae-4055-a87f-45fb8e3b3bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.559921 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-ecffaa17-73ae-4055-a87f-45fb8e3b3bac None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:01.561167 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-ecffaa17-73ae-4055-a87f-45fb8e3b3bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:01.623289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397d686-7f91-45ea-96de-00f0ad52b9d8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:01.633173 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 542/1082] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:43:01.655280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffaa17-73ae-4055-a87f-45fb8e3b3bac None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.658637 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:01.659320 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.659607 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.659889 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.660715 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:01.661086 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:01.662495 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:01.671381 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 541/1083] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:43:01.677873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.765299 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffaa17-73ae-4055-a87f-45fb8e3b3bac None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:01.782846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383a0c-4ac5-4935-b159-5ae555f87471 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:01.797453 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e61889-1112-4f1d-be52-c6767b2a3783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:01.798218 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e61889-1112-4f1d-be52-c6767b2a3783 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.798502 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e61889-1112-4f1d-be52-c6767b2a3783 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.799307 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e61889-1112-4f1d-be52-c6767b2a3783 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.808917 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e61889-1112-4f1d-be52-c6767b2a3783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:01.816056 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa1885-60a0-462a-8cbf-b5f0f7efafdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:01.816925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa1885-60a0-462a-8cbf-b5f0f7efafdb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.817245 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa1885-60a0-462a-8cbf-b5f0f7efafdb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.817531 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa1885-60a0-462a-8cbf-b5f0f7efafdb None admin] PATH_INFO: `/v3/users/45533fa3735c40918eb22e2592fc23aa` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-e9e61889-1112-4f1d-be52-c6767b2a3783 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:01.817651 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:01.818778 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:01.819826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 543/1084] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:43:01.820665 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa1885-60a0-462a-8cbf-b5f0f7efafdb None admin] RBAC: Authorizing `identity:get_user(user_id=45533fa3735c40918eb22e2592fc23aa)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:01.820665 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa1885-60a0-462a-8cbf-b5f0f7efafdb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:01.826396 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 542/1085] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/users/45533fa3735c40918eb22e2592fc23aa => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:43:01.826628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e61889-1112-4f1d-be52-c6767b2a3783 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.827256 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa1885-60a0-462a-8cbf-b5f0f7efafdb None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.918176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e61889-1112-4f1d-be52-c6767b2a3783 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:01.925453 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa1885-60a0-462a-8cbf-b5f0f7efafdb None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:01.950718 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:01.951390 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.951679 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.951963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.952719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:01.953058 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:01.954443 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:01.958127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924f13a-3193-4195-900b-0dc9cdab1a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:01.958988 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924f13a-3193-4195-900b-0dc9cdab1a51 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:01.959714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924f13a-3193-4195-900b-0dc9cdab1a51 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:01.960004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924f13a-3193-4195-900b-0dc9cdab1a51 None admin] PATH_INFO: `/v3/projects/3800f7bfd9e2467c873821f2ef596d98` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:01.961247 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924f13a-3193-4195-900b-0dc9cdab1a51 None admin] RBAC: Authorizing `identity:get_project(project_id=3800f7bfd9e2467c873821f2ef596d98)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:01.962224 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 544/1086] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:01.962672 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924f13a-3193-4195-900b-0dc9cdab1a51 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:01.964374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 543/1087] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/projects/3800f7bfd9e2467c873821f2ef596d98 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:43:01.967936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:01.970028 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924f13a-3193-4195-900b-0dc9cdab1a51 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:02.068317 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924f13a-3193-4195-900b-0dc9cdab1a51 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:02.069658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afb70a-fdf5-496e-a419-f8e3a7abf869 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:02.100713 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbc4d6-3fc6-4ac1-aaa4-c75581367b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:02.101642 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc4d6-3fc6-4ac1-aaa4-c75581367b5d None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:02.101934 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc4d6-3fc6-4ac1-aaa4-c75581367b5d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:02.102048 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:02.102221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc4d6-3fc6-4ac1-aaa4-c75581367b5d None admin] PATH_INFO: `/v3/projects/3800f7bfd9e2467c873821f2ef596d98/users/45533fa3735c40918eb22e2592fc23aa/roles/ebff922955b74c919d58613875326151` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:02.102760 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:02.103054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:02.103342 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:02.104233 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbc4d6-3fc6-4ac1-aaa4-c75581367b5d None admin] RBAC: Authorizing `identity:create_grant(project_id=3800f7bfd9e2467c873821f2ef596d98, user_id=45533fa3735c40918eb22e2592fc23aa, role_id=ebff922955b74c919d58613875326151)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:02.105719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbc4d6-3fc6-4ac1-aaa4-c75581367b5d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:02.114701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:02.116146 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:02.119349 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 544/1088] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:43:01 2022] PUT /identity/v3/projects/3800f7bfd9e2467c873821f2ef596d98/users/45533fa3735c40918eb22e2592fc23aa/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:43:02.133463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:02.134675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:02.135767 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:43:02.135767 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:43:02.135767 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:43:02.135767 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:02.135767 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 545/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:43:02.141137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbc4d6-3fc6-4ac1-aaa4-c75581367b5d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:02.245874 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbc4d6-3fc6-4ac1-aaa4-c75581367b5d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:02.280587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:02.281308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:02.281600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:02.281884 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:02.282628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:02.282943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:02.284904 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:02.298640 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 545/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:02.303326 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:02.401432 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947af06-5cc8-4241-b426-9920f1bfebfa None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:02.434331 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:02.435135 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:02.435429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:02.435716 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] PATH_INFO: `/v3/users/nova` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:02.443075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:02.444553 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:43:02.458851 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:02.459959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:02.461713 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 546/1091] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 9 15:43:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:43:02.465829 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:02.564175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba46f71-c7b9-4aaa-bd0e-205818889b8b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:02.599328 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:02.602038 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:02.602335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:02.602929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:02.603812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:02.604133 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:02.605604 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:02.623843 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 546/1092] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 9 15:43:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:02.630301 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:02.725841 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6937d-ea42-414b-94e6-11027252089c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:02.757480 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fcc84-a7a0-48b4-b3a6-5876a9aaaff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:02.758129 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fcc84-a7a0-48b4-b3a6-5876a9aaaff9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:02.758417 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fcc84-a7a0-48b4-b3a6-5876a9aaaff9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:02.758696 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fcc84-a7a0-48b4-b3a6-5876a9aaaff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:02.823039 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fcc84-a7a0-48b4-b3a6-5876a9aaaff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:02.824494 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03fcc84-a7a0-48b4-b3a6-5876a9aaaff9 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:02.922401 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 547/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:43:02.928160 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:03.024470 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0954d0-a54a-4ff2-bea9-03ae2e9dee26 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:03.056581 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:03.057399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:03.057688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:03.057969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:03.068390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:03.069885 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:43:03.087012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:03.088241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:03.089355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:43:03.089355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:43:03.089355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:43:03.089355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:43:03.089355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 547/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:03.093959 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fcc84-a7a0-48b4-b3a6-5876a9aaaff9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:03.187153 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03fcc84-a7a0-48b4-b3a6-5876a9aaaff9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:03.219238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:03.219893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:03.220176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:03.220455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:03.221506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:03.221817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:03.223186 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:03.235276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 548/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:03.241564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:03.338091 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898587b-4e71-459f-a7b8-b390845f75b4 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:03.371325 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:03.372120 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:03.372416 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:03.372698 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:03.379896 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:03.381380 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:43:03.394937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:03.396155 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:43:03.397567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 548/1096] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:43:03.402070 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:03.498364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fed23-0ed2-47c9-88b0-fa73e06fd507 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:03.530691 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad5a7-754c-4b8a-af41-5200541d9835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:03.531349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad5a7-754c-4b8a-af41-5200541d9835 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:03.531639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad5a7-754c-4b8a-af41-5200541d9835 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:03.531922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad5a7-754c-4b8a-af41-5200541d9835 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:03.532714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad5a7-754c-4b8a-af41-5200541d9835 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:03.533558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad5a7-754c-4b8a-af41-5200541d9835 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:03.535007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad5a7-754c-4b8a-af41-5200541d9835 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:03.546660 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 549/1097] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:03.553260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:03.648545 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5478d9-c181-4019-a0d8-d351d95edeba None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:03.680882 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13a5e0-7a98-463b-bf9a-91e4ef4eea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:03.681558 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13a5e0-7a98-463b-bf9a-91e4ef4eea22 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:03.681844 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13a5e0-7a98-463b-bf9a-91e4ef4eea22 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:03.682121 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13a5e0-7a98-463b-bf9a-91e4ef4eea22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:03.682950 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13a5e0-7a98-463b-bf9a-91e4ef4eea22 None admin] RBAC: Adding query filter params (role.id=97cdf992a64b4a93b8ecef5c96c7dc35, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=ef8546c698f9455cb81264bcee0d2344) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:03.683266 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13a5e0-7a98-463b-bf9a-91e4ef4eea22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:03.684696 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13a5e0-7a98-463b-bf9a-91e4ef4eea22 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:03.690383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 549/1098] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:43:03 2022] GET /identity/v3/role_assignments?user.id=ef8546c698f9455cb81264bcee0d2344&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=97cdf992a64b4a93b8ecef5c96c7dc35 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:43:04.357441 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4f3b-9a44-49fa-93f0-37ea12581fd7 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:04.357949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4f3b-9a44-49fa-93f0-37ea12581fd7 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:04.357949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4f3b-9a44-49fa-93f0-37ea12581fd7 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:04.359329 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 550/1099] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:04.364822 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031fdfa-498d-4874-8446-61bd00216190 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:04.365300 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031fdfa-498d-4874-8446-61bd00216190 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:04.365420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031fdfa-498d-4874-8446-61bd00216190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:04.382883 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-3031fdfa-498d-4874-8446-61bd00216190 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:04.384084 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-3031fdfa-498d-4874-8446-61bd00216190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:04.453937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 550/1100] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:04.477961 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec4f3b-9a44-49fa-93f0-37ea12581fd7 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:04.586131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec4f3b-9a44-49fa-93f0-37ea12581fd7 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:04.620880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57da90a-898b-4505-a5d8-3271ba4c8fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:04.621654 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57da90a-898b-4505-a5d8-3271ba4c8fdd None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:04.621942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57da90a-898b-4505-a5d8-3271ba4c8fdd None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:04.622223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57da90a-898b-4505-a5d8-3271ba4c8fdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:04.632450 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57da90a-898b-4505-a5d8-3271ba4c8fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-a57da90a-898b-4505-a5d8-3271ba4c8fdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:04.640005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:04.641267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:04.642575 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 551/1101] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:43:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:04.647250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031fdfa-498d-4874-8446-61bd00216190 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:04.740480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031fdfa-498d-4874-8446-61bd00216190 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:04.772704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:04.773405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:04.773694 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:04.773975 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:04.774724 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:04.775034 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:04.776390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:04.784122 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 551/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:43:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:04.789658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57da90a-898b-4505-a5d8-3271ba4c8fdd None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:04.893913 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57da90a-898b-4505-a5d8-3271ba4c8fdd None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:04.930437 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584264c9-271b-455a-9152-b38bb0efd2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:04.931314 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584264c9-271b-455a-9152-b38bb0efd2d1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:04.931600 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584264c9-271b-455a-9152-b38bb0efd2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:04.931880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584264c9-271b-455a-9152-b38bb0efd2d1 None admin] PATH_INFO: `/v3/users/45533fa3735c40918eb22e2592fc23aa` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:04.933078 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584264c9-271b-455a-9152-b38bb0efd2d1 None admin] RBAC: Authorizing `identity:get_user(user_id=45533fa3735c40918eb22e2592fc23aa)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:04.934519 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584264c9-271b-455a-9152-b38bb0efd2d1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:04.936155 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 552/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:04 2022] GET /identity/v3/users/45533fa3735c40918eb22e2592fc23aa => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:04.942199 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:05.037361 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8815e-1fd9-45d6-aa1e-be66909df634 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:05.068768 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a15e0-4027-4f7e-8bfc-085b75f32bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:05.069544 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a15e0-4027-4f7e-8bfc-085b75f32bef None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:05.069829 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a15e0-4027-4f7e-8bfc-085b75f32bef None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:05.070202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a15e0-4027-4f7e-8bfc-085b75f32bef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:05.134266 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a15e0-4027-4f7e-8bfc-085b75f32bef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:05.135697 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a15e0-4027-4f7e-8bfc-085b75f32bef None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:05.230263 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 552/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:43:05.236808 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584264c9-271b-455a-9152-b38bb0efd2d1 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:05.332639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584264c9-271b-455a-9152-b38bb0efd2d1 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:05.365165 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1eecf-2354-41b8-b514-da22f444d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:05.366024 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1eecf-2354-41b8-b514-da22f444d015 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:05.366316 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1eecf-2354-41b8-b514-da22f444d015 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:05.366601 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1eecf-2354-41b8-b514-da22f444d015 None admin] PATH_INFO: `/v3/projects/3800f7bfd9e2467c873821f2ef596d98` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:05.367794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1eecf-2354-41b8-b514-da22f444d015 None admin] RBAC: Authorizing `identity:get_project(project_id=3800f7bfd9e2467c873821f2ef596d98)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:05.369276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1eecf-2354-41b8-b514-da22f444d015 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:05.370951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 553/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:05 2022] GET /identity/v3/projects/3800f7bfd9e2467c873821f2ef596d98 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:05.376824 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a15e0-4027-4f7e-8bfc-085b75f32bef None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:05.472334 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a15e0-4027-4f7e-8bfc-085b75f32bef None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:05.504384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0186a38-ecc2-4d26-a1bf-868700b73236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:05.504976 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186a38-ecc2-4d26-a1bf-868700b73236 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:05.505297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186a38-ecc2-4d26-a1bf-868700b73236 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:05.505582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186a38-ecc2-4d26-a1bf-868700b73236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:05.506419 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0186a38-ecc2-4d26-a1bf-868700b73236 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=3800f7bfd9e2467c873821f2ef596d98, user.id=45533fa3735c40918eb22e2592fc23aa) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:05.506737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0186a38-ecc2-4d26-a1bf-868700b73236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:05.508167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0186a38-ecc2-4d26-a1bf-868700b73236 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:05.513827 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 553/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:43:05 2022] GET /identity/v3/role_assignments?user.id=45533fa3735c40918eb22e2592fc23aa&scope.project.id=3800f7bfd9e2467c873821f2ef596d98&role.id=ebff922955b74c919d58613875326151 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:07.759124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de272bf-66e3-4711-93b4-da62b9e4ded0 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:07.759675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de272bf-66e3-4711-93b4-da62b9e4ded0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:07.759675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de272bf-66e3-4711-93b4-da62b9e4ded0 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:07.761108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 554/1107] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:07.766436 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890113c5-e8ee-43a0-a28b-e363f77bfcf1 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:07.766910 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890113c5-e8ee-43a0-a28b-e363f77bfcf1 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:07.767047 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890113c5-e8ee-43a0-a28b-e363f77bfcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:07.784589 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-890113c5-e8ee-43a0-a28b-e363f77bfcf1 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:07.785805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-890113c5-e8ee-43a0-a28b-e363f77bfcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:07.849280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 554/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:07.873232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de272bf-66e3-4711-93b4-da62b9e4ded0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:07.971647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de272bf-66e3-4711-93b4-da62b9e4ded0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:08.003836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23f3d7-43b9-486f-9677-ce941a80e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:08.004629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23f3d7-43b9-486f-9677-ce941a80e542 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:08.004923 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23f3d7-43b9-486f-9677-ce941a80e542 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:08.005244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23f3d7-43b9-486f-9677-ce941a80e542 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:08.006433 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23f3d7-43b9-486f-9677-ce941a80e542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:08.007864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23f3d7-43b9-486f-9677-ce941a80e542 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:08.009589 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 555/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:43:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:43:08.015217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890113c5-e8ee-43a0-a28b-e363f77bfcf1 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:08.108117 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890113c5-e8ee-43a0-a28b-e363f77bfcf1 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:08.139894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f547dc2-a907-4d6c-a504-0786526ef11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:08.140549 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547dc2-a907-4d6c-a504-0786526ef11b None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:08.140835 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547dc2-a907-4d6c-a504-0786526ef11b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:08.141139 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f547dc2-a907-4d6c-a504-0786526ef11b None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:08.141840 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f547dc2-a907-4d6c-a504-0786526ef11b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:08.143854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f547dc2-a907-4d6c-a504-0786526ef11b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:08.163005 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 555/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:43:08 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:43:10.418333 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc62f-b449-432e-bbcb-eb5588c406b0 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:10.418869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc62f-b449-432e-bbcb-eb5588c406b0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:10.418974 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fc62f-b449-432e-bbcb-eb5588c406b0 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:10.420264 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 556/1111] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:10.426198 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb981d-2c64-4f5c-94c1-0a22f8cee97b None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:10.426592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb981d-2c64-4f5c-94c1-0a22f8cee97b None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:10.426756 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb981d-2c64-4f5c-94c1-0a22f8cee97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:10.442924 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-04fb981d-2c64-4f5c-94c1-0a22f8cee97b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:10.444099 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-04fb981d-2c64-4f5c-94c1-0a22f8cee97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:10.507808 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 556/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:10.531396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fc62f-b449-432e-bbcb-eb5588c406b0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:10.633553 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fc62f-b449-432e-bbcb-eb5588c406b0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:10.666066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc03497-aa2d-4577-8120-100f3f82f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:10.666942 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc03497-aa2d-4577-8120-100f3f82f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:10.667236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc03497-aa2d-4577-8120-100f3f82f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:10.667523 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc03497-aa2d-4577-8120-100f3f82f3b7 None admin] PATH_INFO: `/v3/roles/044a2d93445743af91a96573d39f21eb` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:10.669138 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc03497-aa2d-4577-8120-100f3f82f3b7 None admin] RBAC: Authorizing `identity:get_role(role_id=044a2d93445743af91a96573d39f21eb)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:10.670453 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc03497-aa2d-4577-8120-100f3f82f3b7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:10.671845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 557/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:10 2022] GET /identity/v3/roles/044a2d93445743af91a96573d39f21eb => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:10.677814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb981d-2c64-4f5c-94c1-0a22f8cee97b None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:10.772244 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb981d-2c64-4f5c-94c1-0a22f8cee97b None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:10.804783 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254963e-7814-4651-ba4b-aa04f4d2c114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:10.805678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254963e-7814-4651-ba4b-aa04f4d2c114 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:10.805965 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254963e-7814-4651-ba4b-aa04f4d2c114 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:10.806245 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254963e-7814-4651-ba4b-aa04f4d2c114 None admin] PATH_INFO: `/v3/users/54be5805b2dc4404ba5b8b70aef27501` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:10.818637 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254963e-7814-4651-ba4b-aa04f4d2c114 None admin] RBAC: Authorizing `identity:get_user(user_id=54be5805b2dc4404ba5b8b70aef27501)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:10.820070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254963e-7814-4651-ba4b-aa04f4d2c114 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:10.821737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 557/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:10 2022] GET /identity/v3/users/54be5805b2dc4404ba5b8b70aef27501 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:10.827270 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc03497-aa2d-4577-8120-100f3f82f3b7 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:10.922631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc03497-aa2d-4577-8120-100f3f82f3b7 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:10.955644 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f83bb9-64cc-423e-9578-1da7423c2128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:10.956294 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f83bb9-64cc-423e-9578-1da7423c2128 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:10.956584 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f83bb9-64cc-423e-9578-1da7423c2128 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:10.956866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f83bb9-64cc-423e-9578-1da7423c2128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:11.019936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f83bb9-64cc-423e-9578-1da7423c2128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:11.021406 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f83bb9-64cc-423e-9578-1da7423c2128 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:11.113157 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 558/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:43:11.118971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254963e-7814-4651-ba4b-aa04f4d2c114 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:11.216533 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254963e-7814-4651-ba4b-aa04f4d2c114 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:11.249487 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633aaae-5641-4c7a-b193-6895d0de423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:11.250342 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633aaae-5641-4c7a-b193-6895d0de423a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:11.250628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633aaae-5641-4c7a-b193-6895d0de423a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:11.250908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633aaae-5641-4c7a-b193-6895d0de423a None admin] PATH_INFO: `/v3/projects/3800f7bfd9e2467c873821f2ef596d98` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:11.252622 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633aaae-5641-4c7a-b193-6895d0de423a None admin] RBAC: Authorizing `identity:get_project(project_id=3800f7bfd9e2467c873821f2ef596d98)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:11.254083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633aaae-5641-4c7a-b193-6895d0de423a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:11.255779 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 558/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:11 2022] GET /identity/v3/projects/3800f7bfd9e2467c873821f2ef596d98 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:11.261936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f83bb9-64cc-423e-9578-1da7423c2128 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:11.354798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f83bb9-64cc-423e-9578-1da7423c2128 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:11.388702 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7099d-ae1b-4536-91f3-f33ffe2b90e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:11.389384 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7099d-ae1b-4536-91f3-f33ffe2b90e1 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:11.389677 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7099d-ae1b-4536-91f3-f33ffe2b90e1 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:11.389961 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7099d-ae1b-4536-91f3-f33ffe2b90e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:11.390796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7099d-ae1b-4536-91f3-f33ffe2b90e1 None admin] RBAC: Adding query filter params (role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=3800f7bfd9e2467c873821f2ef596d98, user.id=54be5805b2dc4404ba5b8b70aef27501) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:11.391116 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7099d-ae1b-4536-91f3-f33ffe2b90e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:11.393115 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7099d-ae1b-4536-91f3-f33ffe2b90e1 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:11.398411 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 559/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:43:11 2022] GET /identity/v3/role_assignments?user.id=54be5805b2dc4404ba5b8b70aef27501&scope.project.id=3800f7bfd9e2467c873821f2ef596d98&role.id=044a2d93445743af91a96573d39f21eb => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:13.604619 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6baa09-f52e-4f42-b711-df19a3e2561c None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:13.605221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6baa09-f52e-4f42-b711-df19a3e2561c None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:13.605221 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6baa09-f52e-4f42-b711-df19a3e2561c None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:13.606587 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 559/1118] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:13.612382 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a352f-e0d2-4476-a190-3cf330990ba5 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:13.612811 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a352f-e0d2-4476-a190-3cf330990ba5 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:13.612944 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a352f-e0d2-4476-a190-3cf330990ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:13.629955 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-e03a352f-e0d2-4476-a190-3cf330990ba5 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:13.631143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-e03a352f-e0d2-4476-a190-3cf330990ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:13.694017 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 560/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:13.716495 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6baa09-f52e-4f42-b711-df19a3e2561c None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:13.812890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6baa09-f52e-4f42-b711-df19a3e2561c None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:13.844854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037642c2-575c-4d85-81d3-580d4a4fb398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:13.845750 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037642c2-575c-4d85-81d3-580d4a4fb398 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:13.846297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037642c2-575c-4d85-81d3-580d4a4fb398 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:13.846582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037642c2-575c-4d85-81d3-580d4a4fb398 None admin] PATH_INFO: `/v3/roles/044a2d93445743af91a96573d39f21eb` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:13.848061 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037642c2-575c-4d85-81d3-580d4a4fb398 None admin] RBAC: Authorizing `identity:get_role(role_id=044a2d93445743af91a96573d39f21eb)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:13.849456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037642c2-575c-4d85-81d3-580d4a4fb398 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:13.850789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 560/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:13 2022] GET /identity/v3/roles/044a2d93445743af91a96573d39f21eb => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:13.856471 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a352f-e0d2-4476-a190-3cf330990ba5 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:13.951126 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a352f-e0d2-4476-a190-3cf330990ba5 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:13.983696 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229da23-e871-47a7-a071-163475f806aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:13.984570 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229da23-e871-47a7-a071-163475f806aa None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:13.984862 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229da23-e871-47a7-a071-163475f806aa None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:13.985178 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229da23-e871-47a7-a071-163475f806aa None admin] PATH_INFO: `/v3/users/54be5805b2dc4404ba5b8b70aef27501` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:13.986927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229da23-e871-47a7-a071-163475f806aa None admin] RBAC: Authorizing `identity:get_user(user_id=54be5805b2dc4404ba5b8b70aef27501)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:13.988358 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229da23-e871-47a7-a071-163475f806aa None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:13.990029 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 561/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:13 2022] GET /identity/v3/users/54be5805b2dc4404ba5b8b70aef27501 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:13.995835 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037642c2-575c-4d85-81d3-580d4a4fb398 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:14.088497 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037642c2-575c-4d85-81d3-580d4a4fb398 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:14.121095 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c005a-3cb1-4f1e-9062-8598767e44d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:14.121971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c005a-3cb1-4f1e-9062-8598767e44d3 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:14.122256 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c005a-3cb1-4f1e-9062-8598767e44d3 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:14.122531 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c005a-3cb1-4f1e-9062-8598767e44d3 None admin] PATH_INFO: `/v3/projects/3800f7bfd9e2467c873821f2ef596d98` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:14.123705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c005a-3cb1-4f1e-9062-8598767e44d3 None admin] RBAC: Authorizing `identity:get_project(project_id=3800f7bfd9e2467c873821f2ef596d98)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:14.125141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c005a-3cb1-4f1e-9062-8598767e44d3 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:14.126833 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 561/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:13 2022] GET /identity/v3/projects/3800f7bfd9e2467c873821f2ef596d98 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:14.132531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229da23-e871-47a7-a071-163475f806aa None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:14.226829 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229da23-e871-47a7-a071-163475f806aa None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:14.260231 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b924f8b-53ee-4f45-89ce-2cd75eb7c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:14.261153 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b924f8b-53ee-4f45-89ce-2cd75eb7c53e None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:14.261446 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b924f8b-53ee-4f45-89ce-2cd75eb7c53e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:14.261729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b924f8b-53ee-4f45-89ce-2cd75eb7c53e None admin] PATH_INFO: `/v3/projects/3800f7bfd9e2467c873821f2ef596d98/users/54be5805b2dc4404ba5b8b70aef27501/roles/044a2d93445743af91a96573d39f21eb` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:14.263950 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b924f8b-53ee-4f45-89ce-2cd75eb7c53e None admin] RBAC: Authorizing `identity:create_grant(project_id=3800f7bfd9e2467c873821f2ef596d98, user_id=54be5805b2dc4404ba5b8b70aef27501, role_id=044a2d93445743af91a96573d39f21eb)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:14.265410 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b924f8b-53ee-4f45-89ce-2cd75eb7c53e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:14.277820 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 562/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:43:14 2022] PUT /identity/v3/projects/3800f7bfd9e2467c873821f2ef596d98/users/54be5805b2dc4404ba5b8b70aef27501/roles/044a2d93445743af91a96573d39f21eb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:43:16.457340 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459524e-26f3-4ed3-82b1-6f4cd5ac093f None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:16.457940 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459524e-26f3-4ed3-82b1-6f4cd5ac093f None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:16.457940 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459524e-26f3-4ed3-82b1-6f4cd5ac093f None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:16.459268 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 562/1124] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:16.464447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4fbe4-e825-4cef-8c56-539a02e237ce None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:16.464911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4fbe4-e825-4cef-8c56-539a02e237ce None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:16.465024 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4fbe4-e825-4cef-8c56-539a02e237ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:16.481943 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-83a4fbe4-e825-4cef-8c56-539a02e237ce None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:16.483122 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-83a4fbe4-e825-4cef-8c56-539a02e237ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:16.552337 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 563/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:16.575280 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459524e-26f3-4ed3-82b1-6f4cd5ac093f None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:16.677578 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459524e-26f3-4ed3-82b1-6f4cd5ac093f None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:16.709657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e07d3a-796b-49ee-a5ae-d04bc9f5a927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:16.710446 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e07d3a-796b-49ee-a5ae-d04bc9f5a927 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:16.710734 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e07d3a-796b-49ee-a5ae-d04bc9f5a927 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:16.711016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e07d3a-796b-49ee-a5ae-d04bc9f5a927 None admin] PATH_INFO: `/v3/roles/044a2d93445743af91a96573d39f21eb` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:16.712504 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e07d3a-796b-49ee-a5ae-d04bc9f5a927 None admin] RBAC: Authorizing `identity:get_role(role_id=044a2d93445743af91a96573d39f21eb)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:16.713935 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e07d3a-796b-49ee-a5ae-d04bc9f5a927 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:16.715301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 563/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:16 2022] GET /identity/v3/roles/044a2d93445743af91a96573d39f21eb => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:16.721954 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4fbe4-e825-4cef-8c56-539a02e237ce None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:16.812949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4fbe4-e825-4cef-8c56-539a02e237ce None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:16.846150 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473311a6-0d53-4686-ad2c-bac1f0b48581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:16.847021 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473311a6-0d53-4686-ad2c-bac1f0b48581 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:16.847308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473311a6-0d53-4686-ad2c-bac1f0b48581 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:16.847588 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473311a6-0d53-4686-ad2c-bac1f0b48581 None admin] PATH_INFO: `/v3/users/54be5805b2dc4404ba5b8b70aef27501` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:16.848786 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473311a6-0d53-4686-ad2c-bac1f0b48581 None admin] RBAC: Authorizing `identity:get_user(user_id=54be5805b2dc4404ba5b8b70aef27501)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:16.850246 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473311a6-0d53-4686-ad2c-bac1f0b48581 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:16.851866 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 564/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:16 2022] GET /identity/v3/users/54be5805b2dc4404ba5b8b70aef27501 => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:16.857357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e07d3a-796b-49ee-a5ae-d04bc9f5a927 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:16.962473 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e07d3a-796b-49ee-a5ae-d04bc9f5a927 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:16.994562 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e28fba-6b42-451f-8a4b-0d07b9c5d419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:16.995208 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e28fba-6b42-451f-8a4b-0d07b9c5d419 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:16.995502 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e28fba-6b42-451f-8a4b-0d07b9c5d419 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:16.995783 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e28fba-6b42-451f-8a4b-0d07b9c5d419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:17.058928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e28fba-6b42-451f-8a4b-0d07b9c5d419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:17.060362 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e28fba-6b42-451f-8a4b-0d07b9c5d419 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:17.153511 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 564/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:43:17.159952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473311a6-0d53-4686-ad2c-bac1f0b48581 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:17.256043 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473311a6-0d53-4686-ad2c-bac1f0b48581 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:17.288989 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41966b38-3da9-44f5-a6d2-ed528d315665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:17.289868 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41966b38-3da9-44f5-a6d2-ed528d315665 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:17.290160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41966b38-3da9-44f5-a6d2-ed528d315665 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:17.290441 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41966b38-3da9-44f5-a6d2-ed528d315665 None admin] PATH_INFO: `/v3/projects/3800f7bfd9e2467c873821f2ef596d98` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:17.291621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41966b38-3da9-44f5-a6d2-ed528d315665 None admin] RBAC: Authorizing `identity:get_project(project_id=3800f7bfd9e2467c873821f2ef596d98)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:17.293073 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41966b38-3da9-44f5-a6d2-ed528d315665 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:17.294755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 565/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:17 2022] GET /identity/v3/projects/3800f7bfd9e2467c873821f2ef596d98 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:17.300586 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e28fba-6b42-451f-8a4b-0d07b9c5d419 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:17.394134 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e28fba-6b42-451f-8a4b-0d07b9c5d419 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:17.426006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a8604-63af-4400-b85f-2c7610ff5b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:17.426648 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a8604-63af-4400-b85f-2c7610ff5b9d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:17.426933 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a8604-63af-4400-b85f-2c7610ff5b9d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:17.427210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a8604-63af-4400-b85f-2c7610ff5b9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:17.428030 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a8604-63af-4400-b85f-2c7610ff5b9d None admin] RBAC: Adding query filter params (role.id=044a2d93445743af91a96573d39f21eb, scope.project.id=3800f7bfd9e2467c873821f2ef596d98, user.id=54be5805b2dc4404ba5b8b70aef27501) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:17.428345 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a8604-63af-4400-b85f-2c7610ff5b9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:17.429786 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a8604-63af-4400-b85f-2c7610ff5b9d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:17.435323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 565/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:43:17 2022] GET /identity/v3/role_assignments?user.id=54be5805b2dc4404ba5b8b70aef27501&scope.project.id=3800f7bfd9e2467c873821f2ef596d98&role.id=044a2d93445743af91a96573d39f21eb => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:19.680478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf07e6e-b21a-49a1-82b7-00681fe5fadd None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:19.681096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf07e6e-b21a-49a1-82b7-00681fe5fadd None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:19.681096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf07e6e-b21a-49a1-82b7-00681fe5fadd None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:19.682463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 566/1131] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:19.687820 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0bfd0-705c-4d9f-95d7-c95edd944a26 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:19.688240 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0bfd0-705c-4d9f-95d7-c95edd944a26 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:19.688388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0bfd0-705c-4d9f-95d7-c95edd944a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:19.705308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-7ea0bfd0-705c-4d9f-95d7-c95edd944a26 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:19.706492 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-7ea0bfd0-705c-4d9f-95d7-c95edd944a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:19.769878 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 566/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:19.792836 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf07e6e-b21a-49a1-82b7-00681fe5fadd None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:19.902720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf07e6e-b21a-49a1-82b7-00681fe5fadd None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:19.937920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c684b-da37-4cb3-b282-da90cfcd45d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:19.938669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c684b-da37-4cb3-b282-da90cfcd45d6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:19.938962 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c684b-da37-4cb3-b282-da90cfcd45d6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:19.939246 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c684b-da37-4cb3-b282-da90cfcd45d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:19.940440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c684b-da37-4cb3-b282-da90cfcd45d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:19.941915 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c684b-da37-4cb3-b282-da90cfcd45d6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:19.943555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 567/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:43:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:19.949485 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0bfd0-705c-4d9f-95d7-c95edd944a26 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:20.042455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0bfd0-705c-4d9f-95d7-c95edd944a26 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:20.074264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604127b9-ca6e-44c8-b397-e7ca08291334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:20.074912 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604127b9-ca6e-44c8-b397-e7ca08291334 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:20.075200 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604127b9-ca6e-44c8-b397-e7ca08291334 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:20.075481 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604127b9-ca6e-44c8-b397-e7ca08291334 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:20.076190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604127b9-ca6e-44c8-b397-e7ca08291334 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:20.077643 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604127b9-ca6e-44c8-b397-e7ca08291334 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:20.102461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 567/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:43:19 2022] POST /identity/v3/projects => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:43:22.348540 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec55b6-9ae7-4244-a0e5-990d87e16cdc None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:22.349082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec55b6-9ae7-4244-a0e5-990d87e16cdc None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:22.349082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec55b6-9ae7-4244-a0e5-990d87e16cdc None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:22.350431 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 568/1135] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:22.355944 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b65f-8a2c-43ab-81fa-b9c8b5b072be None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:22.356374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b65f-8a2c-43ab-81fa-b9c8b5b072be None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:22.356525 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70b65f-8a2c-43ab-81fa-b9c8b5b072be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:22.373582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-8d70b65f-8a2c-43ab-81fa-b9c8b5b072be None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:22.374775 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-8d70b65f-8a2c-43ab-81fa-b9c8b5b072be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:22.439512 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 568/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:22.463802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec55b6-9ae7-4244-a0e5-990d87e16cdc None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:22.559599 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec55b6-9ae7-4244-a0e5-990d87e16cdc None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:22.591200 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e349ad-7ab2-4dee-a834-298c106962f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:22.591982 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e349ad-7ab2-4dee-a834-298c106962f7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:22.592268 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e349ad-7ab2-4dee-a834-298c106962f7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:22.592547 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e349ad-7ab2-4dee-a834-298c106962f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:22.593729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e349ad-7ab2-4dee-a834-298c106962f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:22.595729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e349ad-7ab2-4dee-a834-298c106962f7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:22.597594 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 569/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:43:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:22.603289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70b65f-8a2c-43ab-81fa-b9c8b5b072be None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:22.696495 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70b65f-8a2c-43ab-81fa-b9c8b5b072be None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:22.729890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450e04f-d753-4bf1-b791-bc875835ecbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:22.730560 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e04f-d753-4bf1-b791-bc875835ecbe None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:22.730859 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e04f-d753-4bf1-b791-bc875835ecbe None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:22.731145 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e04f-d753-4bf1-b791-bc875835ecbe None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:22.731860 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450e04f-d753-4bf1-b791-bc875835ecbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:22.733321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450e04f-d753-4bf1-b791-bc875835ecbe None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:22.752186 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 569/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:43:22 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:43:24.983077 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b46d50-a1bb-41b1-8667-031bdb4b6bd0 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:24.983605 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b46d50-a1bb-41b1-8667-031bdb4b6bd0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:24.983720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b46d50-a1bb-41b1-8667-031bdb4b6bd0 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:24.985032 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 570/1139] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:24.990226 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54373-6a2a-4310-b3fa-af7b3b557027 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:24.990684 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54373-6a2a-4310-b3fa-af7b3b557027 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:24.990805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54373-6a2a-4310-b3fa-af7b3b557027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:25.006949 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-90a54373-6a2a-4310-b3fa-af7b3b557027 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:25.008126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-90a54373-6a2a-4310-b3fa-af7b3b557027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:25.070610 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 570/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:25.093870 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b46d50-a1bb-41b1-8667-031bdb4b6bd0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:25.191366 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b46d50-a1bb-41b1-8667-031bdb4b6bd0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:25.223506 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012e72-b1ee-4718-9d22-ac13ef308fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:25.224289 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012e72-b1ee-4718-9d22-ac13ef308fc9 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:25.224578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012e72-b1ee-4718-9d22-ac13ef308fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:25.224863 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012e72-b1ee-4718-9d22-ac13ef308fc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:25.234629 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34012e72-b1ee-4718-9d22-ac13ef308fc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-34012e72-b1ee-4718-9d22-ac13ef308fc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:25.242258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:25.243673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:25.245089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 571/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:43:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:25.249190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a54373-6a2a-4310-b3fa-af7b3b557027 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:25.342483 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a54373-6a2a-4310-b3fa-af7b3b557027 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:25.374076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:25.374733 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:25.375020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:25.375297 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:25.376037 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:25.376869 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:25.378252 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:25.385873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 571/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:43:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:43:25.391352 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012e72-b1ee-4718-9d22-ac13ef308fc9 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:25.485340 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012e72-b1ee-4718-9d22-ac13ef308fc9 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:25.517172 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a931e-471b-41ba-b6aa-3926027ea1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:25.518025 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a931e-471b-41ba-b6aa-3926027ea1f5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:25.518313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a931e-471b-41ba-b6aa-3926027ea1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:25.518597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a931e-471b-41ba-b6aa-3926027ea1f5 None admin] PATH_INFO: `/v3/users/7591cc72b75848a59e8e06bfc54bc510` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:25.531131 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a931e-471b-41ba-b6aa-3926027ea1f5 None admin] RBAC: Authorizing `identity:get_user(user_id=7591cc72b75848a59e8e06bfc54bc510)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:25.532597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a931e-471b-41ba-b6aa-3926027ea1f5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:25.534279 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 572/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:25 2022] GET /identity/v3/users/7591cc72b75848a59e8e06bfc54bc510 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:25.539871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:25.636635 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bfdba-1870-497c-bd16-1586ed2b2a9a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:25.668343 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81df7-1021-4fc2-913e-fb23576d2413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:25.668984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81df7-1021-4fc2-913e-fb23576d2413 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:25.669312 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81df7-1021-4fc2-913e-fb23576d2413 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:25.669597 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81df7-1021-4fc2-913e-fb23576d2413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:25.734301 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b81df7-1021-4fc2-913e-fb23576d2413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:25.735812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b81df7-1021-4fc2-913e-fb23576d2413 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:25.828206 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 572/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:43:25.834508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a931e-471b-41ba-b6aa-3926027ea1f5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:25.929521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a931e-471b-41ba-b6aa-3926027ea1f5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:25.962726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7722b-6fe4-4815-b8b1-6d7ca4173a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:25.963577 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7722b-6fe4-4815-b8b1-6d7ca4173a7d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:25.963861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7722b-6fe4-4815-b8b1-6d7ca4173a7d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:25.964138 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7722b-6fe4-4815-b8b1-6d7ca4173a7d None admin] PATH_INFO: `/v3/projects/4bedb5a9d9a74d52af3f9e05a490dd24` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:25.965346 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7722b-6fe4-4815-b8b1-6d7ca4173a7d None admin] RBAC: Authorizing `identity:get_project(project_id=4bedb5a9d9a74d52af3f9e05a490dd24)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:25.966783 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7722b-6fe4-4815-b8b1-6d7ca4173a7d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:25.968466 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 573/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:25 2022] GET /identity/v3/projects/4bedb5a9d9a74d52af3f9e05a490dd24 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:25.974807 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81df7-1021-4fc2-913e-fb23576d2413 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:26.076363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81df7-1021-4fc2-913e-fb23576d2413 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:26.110002 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478ecac-88d2-4324-9696-5c14155c5dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:26.111181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478ecac-88d2-4324-9696-5c14155c5dee None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:26.111475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478ecac-88d2-4324-9696-5c14155c5dee None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:26.111760 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478ecac-88d2-4324-9696-5c14155c5dee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:26.112606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478ecac-88d2-4324-9696-5c14155c5dee None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=4bedb5a9d9a74d52af3f9e05a490dd24, user.id=7591cc72b75848a59e8e06bfc54bc510) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:26.112926 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478ecac-88d2-4324-9696-5c14155c5dee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:26.114400 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478ecac-88d2-4324-9696-5c14155c5dee None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:26.119789 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 573/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:43:25 2022] GET /identity/v3/role_assignments?user.id=7591cc72b75848a59e8e06bfc54bc510&scope.project.id=4bedb5a9d9a74d52af3f9e05a490dd24&role.id=ebff922955b74c919d58613875326151 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:43:28.339950 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffda799-4499-408a-8827-cb89710e58c2 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:28.340478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffda799-4499-408a-8827-cb89710e58c2 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:28.340478 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffda799-4499-408a-8827-cb89710e58c2 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:28.341852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 574/1147] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:28.347337 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e7451-b4ea-4ff9-ab4a-fe317596fe0e None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:28.347738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e7451-b4ea-4ff9-ab4a-fe317596fe0e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:28.347893 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e7451-b4ea-4ff9-ab4a-fe317596fe0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:28.364061 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-f35e7451-b4ea-4ff9-ab4a-fe317596fe0e None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:28.365270 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-f35e7451-b4ea-4ff9-ab4a-fe317596fe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:28.427912 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 574/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:28.450189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffda799-4499-408a-8827-cb89710e58c2 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:28.555727 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffda799-4499-408a-8827-cb89710e58c2 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:28.596455 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db47dc-32c9-4396-ac94-a6534422c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:28.597231 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db47dc-32c9-4396-ac94-a6534422c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:28.597519 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db47dc-32c9-4396-ac94-a6534422c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:28.597803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db47dc-32c9-4396-ac94-a6534422c3c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:28.607819 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db47dc-32c9-4396-ac94-a6534422c3c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-f8db47dc-32c9-4396-ac94-a6534422c3c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:28.615996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:28.617308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:28.618381 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 575/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:28.625307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e7451-b4ea-4ff9-ab4a-fe317596fe0e None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:28.718424 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e7451-b4ea-4ff9-ab4a-fe317596fe0e None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:28.750687 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:28.751347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:28.751632 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:28.751912 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:28.752661 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:28.752972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:28.754355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:28.761952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 575/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:43:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:28.768374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db47dc-32c9-4396-ac94-a6534422c3c0 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:28.863616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db47dc-32c9-4396-ac94-a6534422c3c0 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:28.897904 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1511f-e4a7-4751-b739-f30498fbe0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:28.898788 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1511f-e4a7-4751-b739-f30498fbe0b5 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:28.899079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1511f-e4a7-4751-b739-f30498fbe0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:28.899363 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1511f-e4a7-4751-b739-f30498fbe0b5 None admin] PATH_INFO: `/v3/users/7591cc72b75848a59e8e06bfc54bc510` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:28.900565 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1511f-e4a7-4751-b739-f30498fbe0b5 None admin] RBAC: Authorizing `identity:get_user(user_id=7591cc72b75848a59e8e06bfc54bc510)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:28.902026 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1511f-e4a7-4751-b739-f30498fbe0b5 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:28.903701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 576/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:28 2022] GET /identity/v3/users/7591cc72b75848a59e8e06bfc54bc510 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:28.909648 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:29.003801 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f48a5-ff26-43c3-a54e-8b622054fc54 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:29.035321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61884e3c-e232-47f3-9130-60c628c9f852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:29.036162 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61884e3c-e232-47f3-9130-60c628c9f852 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:29.036448 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61884e3c-e232-47f3-9130-60c628c9f852 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:29.036727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61884e3c-e232-47f3-9130-60c628c9f852 None admin] PATH_INFO: `/v3/projects/4bedb5a9d9a74d52af3f9e05a490dd24` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:29.037931 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61884e3c-e232-47f3-9130-60c628c9f852 None admin] RBAC: Authorizing `identity:get_project(project_id=4bedb5a9d9a74d52af3f9e05a490dd24)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:29.039351 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61884e3c-e232-47f3-9130-60c628c9f852 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:29.041010 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 576/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:28 2022] GET /identity/v3/projects/4bedb5a9d9a74d52af3f9e05a490dd24 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:29.046914 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1511f-e4a7-4751-b739-f30498fbe0b5 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:29.143758 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1511f-e4a7-4751-b739-f30498fbe0b5 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:29.175860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c4c6f-f590-4f9c-a388-20d2df58c9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:29.176814 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c4c6f-f590-4f9c-a388-20d2df58c9eb None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:29.177140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c4c6f-f590-4f9c-a388-20d2df58c9eb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:29.177428 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c4c6f-f590-4f9c-a388-20d2df58c9eb None admin] PATH_INFO: `/v3/projects/4bedb5a9d9a74d52af3f9e05a490dd24/users/7591cc72b75848a59e8e06bfc54bc510/roles/ebff922955b74c919d58613875326151` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:29.179403 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c4c6f-f590-4f9c-a388-20d2df58c9eb None admin] RBAC: Authorizing `identity:create_grant(project_id=4bedb5a9d9a74d52af3f9e05a490dd24, user_id=7591cc72b75848a59e8e06bfc54bc510, role_id=ebff922955b74c919d58613875326151)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:29.180834 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c4c6f-f590-4f9c-a388-20d2df58c9eb None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:29.194260 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 577/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:43:29 2022] PUT /identity/v3/projects/4bedb5a9d9a74d52af3f9e05a490dd24/users/7591cc72b75848a59e8e06bfc54bc510/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:43:31.393558 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db72ed-d3a0-4d1b-aebe-659744c3a0ec None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:31.394089 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db72ed-d3a0-4d1b-aebe-659744c3a0ec None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:31.394089 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db72ed-d3a0-4d1b-aebe-659744c3a0ec None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:31.395421 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 577/1154] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:31.400754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a279d-d21c-41af-a393-423688fd8a4d None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:31.401189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a279d-d21c-41af-a393-423688fd8a4d None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:31.401374 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a279d-d21c-41af-a393-423688fd8a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:31.418253 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-cc5a279d-d21c-41af-a393-423688fd8a4d None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:31.419445 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-cc5a279d-d21c-41af-a393-423688fd8a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:31.482884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 578/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:31.506202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db72ed-d3a0-4d1b-aebe-659744c3a0ec None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:31.603739 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db72ed-d3a0-4d1b-aebe-659744c3a0ec None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:31.634919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9405a-7d8f-4708-8d12-51527a92c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:31.635693 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9405a-7d8f-4708-8d12-51527a92c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:31.635979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9405a-7d8f-4708-8d12-51527a92c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:31.636258 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9405a-7d8f-4708-8d12-51527a92c9d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:31.646871 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9405a-7d8f-4708-8d12-51527a92c9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-69b9405a-7d8f-4708-8d12-51527a92c9d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:31.654405 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:31.655676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:43:31.657011 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 578/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:43:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:31.662095 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a279d-d21c-41af-a393-423688fd8a4d None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:31.753488 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a279d-d21c-41af-a393-423688fd8a4d None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:31.785684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:31.786338 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:31.786626 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:31.786907 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:31.787648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:31.787958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:31.789341 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:31.796903 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 579/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:43:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:31.802438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9405a-7d8f-4708-8d12-51527a92c9d5 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:31.897503 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9405a-7d8f-4708-8d12-51527a92c9d5 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:31.930242 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53415edb-3779-4819-93d0-cb7b127dc181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:31.931101 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415edb-3779-4819-93d0-cb7b127dc181 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:31.931386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415edb-3779-4819-93d0-cb7b127dc181 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:31.931662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53415edb-3779-4819-93d0-cb7b127dc181 None admin] PATH_INFO: `/v3/users/7591cc72b75848a59e8e06bfc54bc510` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:31.932834 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53415edb-3779-4819-93d0-cb7b127dc181 None admin] RBAC: Authorizing `identity:get_user(user_id=7591cc72b75848a59e8e06bfc54bc510)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:31.934287 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53415edb-3779-4819-93d0-cb7b127dc181 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:31.935886 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 579/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:31 2022] GET /identity/v3/users/7591cc72b75848a59e8e06bfc54bc510 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:31.944095 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:32.043737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f90af-3bd5-4b97-b76b-1d5a58089b15 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:32.074889 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4b4d6-4ad4-4e34-afbb-e9963ce0dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:32.075527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4b4d6-4ad4-4e34-afbb-e9963ce0dc97 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:32.075814 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4b4d6-4ad4-4e34-afbb-e9963ce0dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:32.076092 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b4b4d6-4ad4-4e34-afbb-e9963ce0dc97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:32.138765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4b4d6-4ad4-4e34-afbb-e9963ce0dc97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:32.140198 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b4b4d6-4ad4-4e34-afbb-e9963ce0dc97 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:32.231312 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 580/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:43:32.237726 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53415edb-3779-4819-93d0-cb7b127dc181 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:32.342853 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53415edb-3779-4819-93d0-cb7b127dc181 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:32.375212 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e54bf-99e1-4fe6-812d-2cba75811630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:32.376627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e54bf-99e1-4fe6-812d-2cba75811630 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:32.376913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e54bf-99e1-4fe6-812d-2cba75811630 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:32.377227 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e54bf-99e1-4fe6-812d-2cba75811630 None admin] PATH_INFO: `/v3/projects/4bedb5a9d9a74d52af3f9e05a490dd24` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:32.378413 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e54bf-99e1-4fe6-812d-2cba75811630 None admin] RBAC: Authorizing `identity:get_project(project_id=4bedb5a9d9a74d52af3f9e05a490dd24)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:32.379843 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e54bf-99e1-4fe6-812d-2cba75811630 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:32.381558 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 580/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:32 2022] GET /identity/v3/projects/4bedb5a9d9a74d52af3f9e05a490dd24 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:32.387674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4b4d6-4ad4-4e34-afbb-e9963ce0dc97 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:32.481047 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b4b4d6-4ad4-4e34-afbb-e9963ce0dc97 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:32.513616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c593e5c-9603-476f-b114-5fac30323e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:32.514207 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593e5c-9603-476f-b114-5fac30323e1f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:32.514492 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593e5c-9603-476f-b114-5fac30323e1f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:32.514770 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593e5c-9603-476f-b114-5fac30323e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:32.515596 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c593e5c-9603-476f-b114-5fac30323e1f None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=4bedb5a9d9a74d52af3f9e05a490dd24, user.id=7591cc72b75848a59e8e06bfc54bc510) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:32.515910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c593e5c-9603-476f-b114-5fac30323e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:32.517375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c593e5c-9603-476f-b114-5fac30323e1f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:32.523056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 581/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:43:32 2022] GET /identity/v3/role_assignments?user.id=7591cc72b75848a59e8e06bfc54bc510&scope.project.id=4bedb5a9d9a74d52af3f9e05a490dd24&role.id=ebff922955b74c919d58613875326151 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:34.768239 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dd63-9712-4b35-89ac-f7c8c2fb12f7 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:34.768811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dd63-9712-4b35-89ac-f7c8c2fb12f7 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:34.768811 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8dd63-9712-4b35-89ac-f7c8c2fb12f7 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:34.770204 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 581/1162] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:34.775508 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa0c54-2ae3-421c-81b2-b722563261f0 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:34.775940 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa0c54-2ae3-421c-81b2-b722563261f0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:34.776070 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa0c54-2ae3-421c-81b2-b722563261f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:34.792222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-3dfa0c54-2ae3-421c-81b2-b722563261f0 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:34.793440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-3dfa0c54-2ae3-421c-81b2-b722563261f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:34.856228 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 582/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:34.879024 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8dd63-9712-4b35-89ac-f7c8c2fb12f7 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:34.977606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8dd63-9712-4b35-89ac-f7c8c2fb12f7 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:35.010960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca18b7-a792-4ba2-8e13-9c81cfc76a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:35.011662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca18b7-a792-4ba2-8e13-9c81cfc76a05 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:35.012050 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca18b7-a792-4ba2-8e13-9c81cfc76a05 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:35.012456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca18b7-a792-4ba2-8e13-9c81cfc76a05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:35.077964 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca18b7-a792-4ba2-8e13-9c81cfc76a05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:35.079404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca18b7-a792-4ba2-8e13-9c81cfc76a05 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:35.171911 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 582/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 09 15:43:35.178491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa0c54-2ae3-421c-81b2-b722563261f0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:35.272091 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa0c54-2ae3-421c-81b2-b722563261f0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:35.309695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ae777-d87c-489d-8ccd-871d7bdcacb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:35.310517 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ae777-d87c-489d-8ccd-871d7bdcacb4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:35.310809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ae777-d87c-489d-8ccd-871d7bdcacb4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:35.311094 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ae777-d87c-489d-8ccd-871d7bdcacb4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:35.318077 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ae777-d87c-489d-8ccd-871d7bdcacb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:35.319531 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ae777-d87c-489d-8ccd-871d7bdcacb4 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-f40ae777-d87c-489d-8ccd-871d7bdcacb4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:35.334159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:35.335512 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:35.336638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 583/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:43:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:35.341347 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca18b7-a792-4ba2-8e13-9c81cfc76a05 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:35.435361 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca18b7-a792-4ba2-8e13-9c81cfc76a05 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:35.467361 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ea73e-92bc-4eb9-8c06-6f2cd3929261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:35.468009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ea73e-92bc-4eb9-8c06-6f2cd3929261 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:35.468295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ea73e-92bc-4eb9-8c06-6f2cd3929261 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:35.468571 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ea73e-92bc-4eb9-8c06-6f2cd3929261 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:35.469317 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ea73e-92bc-4eb9-8c06-6f2cd3929261 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:35.470240 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ea73e-92bc-4eb9-8c06-6f2cd3929261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:35.471861 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ea73e-92bc-4eb9-8c06-6f2cd3929261 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:35.478737 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 583/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 9 15:43:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:37.673281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659310d4-fa2e-4d05-9bbf-d795b7e60090 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:37.673901 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659310d4-fa2e-4d05-9bbf-d795b7e60090 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:37.673901 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659310d4-fa2e-4d05-9bbf-d795b7e60090 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:37.675204 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 584/1167] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:37.680371 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30309a2-6a23-4ff2-8e1e-dde1e261328b None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:37.680819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30309a2-6a23-4ff2-8e1e-dde1e261328b None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:37.680933 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30309a2-6a23-4ff2-8e1e-dde1e261328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:37.697902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-e30309a2-6a23-4ff2-8e1e-dde1e261328b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:37.699090 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-e30309a2-6a23-4ff2-8e1e-dde1e261328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:37.761939 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 584/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:37.785177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659310d4-fa2e-4d05-9bbf-d795b7e60090 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:37.882168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659310d4-fa2e-4d05-9bbf-d795b7e60090 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:37.915667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733ec83-994b-45bf-95bd-d4facb7078c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:37.916344 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733ec83-994b-45bf-95bd-d4facb7078c6 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:37.916636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733ec83-994b-45bf-95bd-d4facb7078c6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:37.917437 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733ec83-994b-45bf-95bd-d4facb7078c6 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:37.917697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733ec83-994b-45bf-95bd-d4facb7078c6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:37.919076 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733ec83-994b-45bf-95bd-d4facb7078c6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:37.934280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 585/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Mon May 9 15:43:37 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:43:40.145550 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efb2a0-3e32-4ca7-b645-6dba2eb923f8 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:40.146112 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efb2a0-3e32-4ca7-b645-6dba2eb923f8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:40.146112 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efb2a0-3e32-4ca7-b645-6dba2eb923f8 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:40.147483 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 585/1170] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:40.152611 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b495ca3-e35c-4ef8-ab13-b923180ca45c None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:40.153095 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b495ca3-e35c-4ef8-ab13-b923180ca45c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:40.153213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b495ca3-e35c-4ef8-ab13-b923180ca45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:40.170087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-4b495ca3-e35c-4ef8-ab13-b923180ca45c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:40.171232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-4b495ca3-e35c-4ef8-ab13-b923180ca45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:40.234903 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 586/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:40.258020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efb2a0-3e32-4ca7-b645-6dba2eb923f8 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:40.355104 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efb2a0-3e32-4ca7-b645-6dba2eb923f8 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:40.387564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043518e1-4adc-4c72-8c84-29aef7daa543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:40.388412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043518e1-4adc-4c72-8c84-29aef7daa543 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:40.388702 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043518e1-4adc-4c72-8c84-29aef7daa543 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:40.388983 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043518e1-4adc-4c72-8c84-29aef7daa543 None admin] PATH_INFO: `/v3/domains/faf575be055f45c6ad35663e15eaead0` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:40.399548 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043518e1-4adc-4c72-8c84-29aef7daa543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=faf575be055f45c6ad35663e15eaead0)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:40.400984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043518e1-4adc-4c72-8c84-29aef7daa543 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:40.402604 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 586/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 9 15:43:40 2022] GET /identity/v3/domains/faf575be055f45c6ad35663e15eaead0 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:40.408281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b495ca3-e35c-4ef8-ab13-b923180ca45c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:40.503564 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b495ca3-e35c-4ef8-ab13-b923180ca45c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:40.535391 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744fe6b-a6ec-4361-909f-9c0fcbdafc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:40.536051 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744fe6b-a6ec-4361-909f-9c0fcbdafc59 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:40.536340 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744fe6b-a6ec-4361-909f-9c0fcbdafc59 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:40.536620 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744fe6b-a6ec-4361-909f-9c0fcbdafc59 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:40.537361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744fe6b-a6ec-4361-909f-9c0fcbdafc59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:40.538791 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744fe6b-a6ec-4361-909f-9c0fcbdafc59 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:40.562823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 587/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:43:40 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:43:42.795069 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde637f6-0c3f-4f23-a430-777598a2cf9b None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:42.795663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde637f6-0c3f-4f23-a430-777598a2cf9b None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:42.795663 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde637f6-0c3f-4f23-a430-777598a2cf9b None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:42.796990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 587/1174] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:42.802406 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28f3eb-1e39-4f92-b9e0-80746b224d25 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:42.802854 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28f3eb-1e39-4f92-b9e0-80746b224d25 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:42.802972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28f3eb-1e39-4f92-b9e0-80746b224d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:42.819181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-4e28f3eb-1e39-4f92-b9e0-80746b224d25 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:42.820458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-4e28f3eb-1e39-4f92-b9e0-80746b224d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:42.883473 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 588/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:42.908767 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde637f6-0c3f-4f23-a430-777598a2cf9b None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:43.005613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde637f6-0c3f-4f23-a430-777598a2cf9b None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:43.038467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2390e-5e99-4590-af40-3af69d59468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:43.039326 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2390e-5e99-4590-af40-3af69d59468c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:43.039616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2390e-5e99-4590-af40-3af69d59468c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:43.039899 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2390e-5e99-4590-af40-3af69d59468c None admin] PATH_INFO: `/v3/domains/faf575be055f45c6ad35663e15eaead0` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:43.041109 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2390e-5e99-4590-af40-3af69d59468c None admin] RBAC: Authorizing `identity:get_domain(domain_id=faf575be055f45c6ad35663e15eaead0)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:43.042535 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2390e-5e99-4590-af40-3af69d59468c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:43.044183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 588/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 9 15:43:42 2022] GET /identity/v3/domains/faf575be055f45c6ad35663e15eaead0 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:43.050037 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28f3eb-1e39-4f92-b9e0-80746b224d25 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:43.142682 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28f3eb-1e39-4f92-b9e0-80746b224d25 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:43.173917 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f174c-73e3-407c-ba37-b407678c724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:43.174574 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f174c-73e3-407c-ba37-b407678c724c None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:43.174861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f174c-73e3-407c-ba37-b407678c724c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:43.175143 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f174c-73e3-407c-ba37-b407678c724c None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:43.175845 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f174c-73e3-407c-ba37-b407678c724c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:43.177284 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f174c-73e3-407c-ba37-b407678c724c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:43.197891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 589/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:43:43 2022] POST /identity/v3/users => generated 311 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:43:45.455883 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c34510-35a9-480e-8eb7-52b2e88ab101 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:45.456433 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c34510-35a9-480e-8eb7-52b2e88ab101 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:45.456433 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c34510-35a9-480e-8eb7-52b2e88ab101 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:45.458017 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 589/1178] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:45.463141 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618a84-4be0-482e-a0e6-ed612e2f07ff None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:45.463581 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618a84-4be0-482e-a0e6-ed612e2f07ff None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:45.463704 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618a84-4be0-482e-a0e6-ed612e2f07ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:45.480861 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-7b618a84-4be0-482e-a0e6-ed612e2f07ff None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:45.482081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-7b618a84-4be0-482e-a0e6-ed612e2f07ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:45.546310 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 590/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:45.570078 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c34510-35a9-480e-8eb7-52b2e88ab101 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:45.667842 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c34510-35a9-480e-8eb7-52b2e88ab101 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:45.699870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171ab9e-3a2b-4fd5-87ac-02fa93ecf02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:45.700652 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171ab9e-3a2b-4fd5-87ac-02fa93ecf02e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:45.700944 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171ab9e-3a2b-4fd5-87ac-02fa93ecf02e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:45.701261 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171ab9e-3a2b-4fd5-87ac-02fa93ecf02e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:45.711583 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171ab9e-3a2b-4fd5-87ac-02fa93ecf02e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-0171ab9e-3a2b-4fd5-87ac-02fa93ecf02e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:45.719854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:45.721498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:43:45.722608 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 590/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:43:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:45.727354 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b618a84-4be0-482e-a0e6-ed612e2f07ff None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:45.822111 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b618a84-4be0-482e-a0e6-ed612e2f07ff None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:45.856133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:45.856797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:45.857113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:45.857395 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:45.858147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:45.858459 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:45.859843 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:45.867616 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 591/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:43:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:45.873488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171ab9e-3a2b-4fd5-87ac-02fa93ecf02e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:45.967723 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171ab9e-3a2b-4fd5-87ac-02fa93ecf02e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:46.000212 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608cb11-bc05-49b5-8623-cf53625b7847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:46.001122 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608cb11-bc05-49b5-8623-cf53625b7847 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:46.001411 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608cb11-bc05-49b5-8623-cf53625b7847 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:46.001692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608cb11-bc05-49b5-8623-cf53625b7847 None admin] PATH_INFO: `/v3/users/911bb5537c0d49cfb17fecfb47b663d4` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:46.014069 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608cb11-bc05-49b5-8623-cf53625b7847 None admin] RBAC: Authorizing `identity:get_user(user_id=911bb5537c0d49cfb17fecfb47b663d4)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:46.015499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608cb11-bc05-49b5-8623-cf53625b7847 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:46.017706 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 591/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:45 2022] GET /identity/v3/users/911bb5537c0d49cfb17fecfb47b663d4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:46.023394 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:46.119796 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb3db7-a0cc-45cc-9403-0ac4941b4103 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:46.152168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b69ac-bba5-4ca2-9dab-d03c3f85d6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:46.152817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b69ac-bba5-4ca2-9dab-d03c3f85d6a4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:46.153135 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b69ac-bba5-4ca2-9dab-d03c3f85d6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:46.153423 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b69ac-bba5-4ca2-9dab-d03c3f85d6a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:46.215819 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b69ac-bba5-4ca2-9dab-d03c3f85d6a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:46.217276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b69ac-bba5-4ca2-9dab-d03c3f85d6a4 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:46.308016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 592/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:43:46.314430 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608cb11-bc05-49b5-8623-cf53625b7847 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:46.409509 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608cb11-bc05-49b5-8623-cf53625b7847 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:46.442093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b092e91-7e5a-440c-a407-cffe0f0c8d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:46.442946 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b092e91-7e5a-440c-a407-cffe0f0c8d7b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:46.443240 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b092e91-7e5a-440c-a407-cffe0f0c8d7b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:46.443526 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b092e91-7e5a-440c-a407-cffe0f0c8d7b None admin] PATH_INFO: `/v3/projects/a071d94ffbd748f09382ac27fe4acd22` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:46.444719 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b092e91-7e5a-440c-a407-cffe0f0c8d7b None admin] RBAC: Authorizing `identity:get_project(project_id=a071d94ffbd748f09382ac27fe4acd22)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:46.446172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b092e91-7e5a-440c-a407-cffe0f0c8d7b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:46.447841 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 592/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:46 2022] GET /identity/v3/projects/a071d94ffbd748f09382ac27fe4acd22 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:46.454430 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b69ac-bba5-4ca2-9dab-d03c3f85d6a4 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:46.549481 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b69ac-bba5-4ca2-9dab-d03c3f85d6a4 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:46.582086 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43754-22d3-4d8e-8769-8b5fc7788113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:46.582677 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43754-22d3-4d8e-8769-8b5fc7788113 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:46.583108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43754-22d3-4d8e-8769-8b5fc7788113 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:46.583402 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43754-22d3-4d8e-8769-8b5fc7788113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:46.584238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43754-22d3-4d8e-8769-8b5fc7788113 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=a071d94ffbd748f09382ac27fe4acd22, user.id=911bb5537c0d49cfb17fecfb47b663d4) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:46.584558 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43754-22d3-4d8e-8769-8b5fc7788113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:46.586009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43754-22d3-4d8e-8769-8b5fc7788113 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:46.591230 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 593/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:43:46 2022] GET /identity/v3/role_assignments?user.id=911bb5537c0d49cfb17fecfb47b663d4&scope.project.id=a071d94ffbd748f09382ac27fe4acd22&role.id=ebff922955b74c919d58613875326151 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:48.785027 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d016-9cc0-4e86-8e39-1ac841dbf6f2 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:48.785705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d016-9cc0-4e86-8e39-1ac841dbf6f2 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:48.785705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d016-9cc0-4e86-8e39-1ac841dbf6f2 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:48.786978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 593/1186] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:48.792149 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb44d97-9c9b-4c3e-b5e5-bccb4d08b16f None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:48.792567 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb44d97-9c9b-4c3e-b5e5-bccb4d08b16f None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:48.792725 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb44d97-9c9b-4c3e-b5e5-bccb4d08b16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:48.809633 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-1eb44d97-9c9b-4c3e-b5e5-bccb4d08b16f None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:48.810834 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-1eb44d97-9c9b-4c3e-b5e5-bccb4d08b16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:48.874357 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 594/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:48.897255 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77d016-9cc0-4e86-8e39-1ac841dbf6f2 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:48.994130 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77d016-9cc0-4e86-8e39-1ac841dbf6f2 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:49.025785 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c3adb-4dfc-4206-906e-5b49d1b746b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:49.026555 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c3adb-4dfc-4206-906e-5b49d1b746b9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:49.026839 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c3adb-4dfc-4206-906e-5b49d1b746b9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:49.027116 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c3adb-4dfc-4206-906e-5b49d1b746b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:49.036738 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c3adb-4dfc-4206-906e-5b49d1b746b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-288c3adb-4dfc-4206-906e-5b49d1b746b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:49.044167 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:49.045458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:43:49.046736 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 594/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:43:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:49.052093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb44d97-9c9b-4c3e-b5e5-bccb4d08b16f None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:49.153388 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb44d97-9c9b-4c3e-b5e5-bccb4d08b16f None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:49.186124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:49.186792 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:49.187080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:49.187364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:49.188142 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:49.188454 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:49.189846 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:49.197784 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 595/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:43:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:49.203854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c3adb-4dfc-4206-906e-5b49d1b746b9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:49.299434 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c3adb-4dfc-4206-906e-5b49d1b746b9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:49.330903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894271a-3102-48c9-a00a-8b1355b65e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:49.331754 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894271a-3102-48c9-a00a-8b1355b65e96 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:49.332048 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894271a-3102-48c9-a00a-8b1355b65e96 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:49.332332 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894271a-3102-48c9-a00a-8b1355b65e96 None admin] PATH_INFO: `/v3/users/911bb5537c0d49cfb17fecfb47b663d4` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:49.333566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894271a-3102-48c9-a00a-8b1355b65e96 None admin] RBAC: Authorizing `identity:get_user(user_id=911bb5537c0d49cfb17fecfb47b663d4)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:49.335000 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894271a-3102-48c9-a00a-8b1355b65e96 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:49.336596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 595/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:49 2022] GET /identity/v3/users/911bb5537c0d49cfb17fecfb47b663d4 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:49.342114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:49.441029 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ff5f-b184-49e5-a8fb-58f674c48f9e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:49.472941 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931fb64-2bcc-48dc-ae55-2079677df0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:49.473828 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931fb64-2bcc-48dc-ae55-2079677df0b4 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:49.474120 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931fb64-2bcc-48dc-ae55-2079677df0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:49.474403 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931fb64-2bcc-48dc-ae55-2079677df0b4 None admin] PATH_INFO: `/v3/projects/a071d94ffbd748f09382ac27fe4acd22` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:49.475583 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931fb64-2bcc-48dc-ae55-2079677df0b4 None admin] RBAC: Authorizing `identity:get_project(project_id=a071d94ffbd748f09382ac27fe4acd22)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:49.477018 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931fb64-2bcc-48dc-ae55-2079677df0b4 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:49.478743 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 596/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:49 2022] GET /identity/v3/projects/a071d94ffbd748f09382ac27fe4acd22 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:49.485395 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894271a-3102-48c9-a00a-8b1355b65e96 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:49.580133 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894271a-3102-48c9-a00a-8b1355b65e96 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:49.612295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263bd2c2-3e68-4691-87cb-0cae3c4d3fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:49.613207 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd2c2-3e68-4691-87cb-0cae3c4d3fee None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:49.613499 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd2c2-3e68-4691-87cb-0cae3c4d3fee None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:49.613783 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd2c2-3e68-4691-87cb-0cae3c4d3fee None admin] PATH_INFO: `/v3/projects/a071d94ffbd748f09382ac27fe4acd22/users/911bb5537c0d49cfb17fecfb47b663d4/roles/ebff922955b74c919d58613875326151` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:49.615755 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bd2c2-3e68-4691-87cb-0cae3c4d3fee None admin] RBAC: Authorizing `identity:create_grant(project_id=a071d94ffbd748f09382ac27fe4acd22, user_id=911bb5537c0d49cfb17fecfb47b663d4, role_id=ebff922955b74c919d58613875326151)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:49.617209 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263bd2c2-3e68-4691-87cb-0cae3c4d3fee None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:49.629931 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 596/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:43:49 2022] PUT /identity/v3/projects/a071d94ffbd748f09382ac27fe4acd22/users/911bb5537c0d49cfb17fecfb47b663d4/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:43:51.865537 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551da671-d01b-4d1d-a8cb-3a5970418272 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:51.866108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551da671-d01b-4d1d-a8cb-3a5970418272 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:51.866108 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551da671-d01b-4d1d-a8cb-3a5970418272 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:51.867722 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 597/1193] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:43:51.873177 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f30f24-1b4b-4800-9429-6f30842086f8 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:51.873629 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f30f24-1b4b-4800-9429-6f30842086f8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:51.873731 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f30f24-1b4b-4800-9429-6f30842086f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:51.892289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-b3f30f24-1b4b-4800-9429-6f30842086f8 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:43:51.893525 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-b3f30f24-1b4b-4800-9429-6f30842086f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:43:51.959386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 597/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:43:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:43:51.982349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551da671-d01b-4d1d-a8cb-3a5970418272 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:52.085935 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551da671-d01b-4d1d-a8cb-3a5970418272 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:52.118147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a597-c751-413a-8dc0-890754e5e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:52.118875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a597-c751-413a-8dc0-890754e5e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:52.119162 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a597-c751-413a-8dc0-890754e5e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:52.119440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a597-c751-413a-8dc0-890754e5e6b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:52.129640 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177a597-c751-413a-8dc0-890754e5e6b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-9177a597-c751-413a-8dc0-890754e5e6b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:43:52.137180 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:43:52.138521 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:43:52.139550 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 598/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:43:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:43:52.144412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f30f24-1b4b-4800-9429-6f30842086f8 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:52.239259 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f30f24-1b4b-4800-9429-6f30842086f8 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:52.271114 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:52.271773 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:52.272060 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:52.272348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:52.273127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:52.273443 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:52.274805 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:52.283000 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 598/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:43:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:52.288848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a597-c751-413a-8dc0-890754e5e6b2 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:52.389262 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a597-c751-413a-8dc0-890754e5e6b2 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:52.432733 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb4cc1-d083-40b7-ba7c-66ceee447b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:52.434106 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb4cc1-d083-40b7-ba7c-66ceee447b44 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:52.434564 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb4cc1-d083-40b7-ba7c-66ceee447b44 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:52.434932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb4cc1-d083-40b7-ba7c-66ceee447b44 None admin] PATH_INFO: `/v3/users/911bb5537c0d49cfb17fecfb47b663d4` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:52.436179 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb4cc1-d083-40b7-ba7c-66ceee447b44 None admin] RBAC: Authorizing `identity:get_user(user_id=911bb5537c0d49cfb17fecfb47b663d4)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:52.437770 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb4cc1-d083-40b7-ba7c-66ceee447b44 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:52.439960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 599/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 9 15:43:52 2022] GET /identity/v3/users/911bb5537c0d49cfb17fecfb47b663d4 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:52.445642 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:52.541355 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67857c4e-18b4-4e85-ba3b-690a9e7aff98 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:52.573766 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369b579-e653-496c-b651-2b6358225543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:52.574406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369b579-e653-496c-b651-2b6358225543 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:52.574693 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369b579-e653-496c-b651-2b6358225543 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:52.574972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3369b579-e653-496c-b651-2b6358225543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:52.639053 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369b579-e653-496c-b651-2b6358225543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:52.640559 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3369b579-e653-496c-b651-2b6358225543 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:52.733940 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 599/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:43:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:43:52.739891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb4cc1-d083-40b7-ba7c-66ceee447b44 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:52.837013 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb4cc1-d083-40b7-ba7c-66ceee447b44 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:52.868687 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b17c-cd51-45a4-a09e-f47f22ccf35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:52.869579 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b17c-cd51-45a4-a09e-f47f22ccf35d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:52.869864 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b17c-cd51-45a4-a09e-f47f22ccf35d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:52.870145 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b17c-cd51-45a4-a09e-f47f22ccf35d None admin] PATH_INFO: `/v3/projects/a071d94ffbd748f09382ac27fe4acd22` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:52.871315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b17c-cd51-45a4-a09e-f47f22ccf35d None admin] RBAC: Authorizing `identity:get_project(project_id=a071d94ffbd748f09382ac27fe4acd22)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:52.872742 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b17c-cd51-45a4-a09e-f47f22ccf35d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:52.874465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 600/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 9 15:43:52 2022] GET /identity/v3/projects/a071d94ffbd748f09382ac27fe4acd22 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:43:52.881020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369b579-e653-496c-b651-2b6358225543 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:43:52.976588 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3369b579-e653-496c-b651-2b6358225543 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:43:53.009474 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0e511-0229-4169-926a-a126dd81b022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:43:53.010122 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0e511-0229-4169-926a-a126dd81b022 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:43:53.010414 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0e511-0229-4169-926a-a126dd81b022 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:43:53.010699 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0e511-0229-4169-926a-a126dd81b022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:43:53.011541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0e511-0229-4169-926a-a126dd81b022 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=a071d94ffbd748f09382ac27fe4acd22, user.id=911bb5537c0d49cfb17fecfb47b663d4) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:43:53.011863 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0e511-0229-4169-926a-a126dd81b022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:43:53.013316 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0e511-0229-4169-926a-a126dd81b022 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:43:53.019602 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 600/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:43:52 2022] GET /identity/v3/role_assignments?user.id=911bb5537c0d49cfb17fecfb47b663d4&scope.project.id=a071d94ffbd748f09382ac27fe4acd22&role.id=ebff922955b74c919d58613875326151 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:44:51.549276 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb9a99-8fcc-4c04-b327-b377113850a2 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:51.549960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb9a99-8fcc-4c04-b327-b377113850a2 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:51.549960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb9a99-8fcc-4c04-b327-b377113850a2 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:51.551223 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 601/1201] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:44:51.557458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c04c-d557-4568-bb64-187bc61e33f3 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:51.557943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c04c-d557-4568-bb64-187bc61e33f3 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:51.558192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3c04c-d557-4568-bb64-187bc61e33f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:51.576143 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-a9c3c04c-d557-4568-bb64-187bc61e33f3 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:44:51.577544 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-a9c3c04c-d557-4568-bb64-187bc61e33f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:44:51.650289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 601/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:44:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:44:51.675617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb9a99-8fcc-4c04-b327-b377113850a2 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:51.786281 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb9a99-8fcc-4c04-b327-b377113850a2 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:51.818621 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:51.819349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:51.819636 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:51.819911 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:51.831089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:51.832552 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:44:51.857125 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:44:51.858429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:44:51.859726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:44:51.859726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:44:51.859726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:44:51.859726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:44:51.859726 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 602/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:44:51.866020 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c04c-d557-4568-bb64-187bc61e33f3 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:51.979522 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3c04c-d557-4568-bb64-187bc61e33f3 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:52.022958 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a97f9-5883-45ea-a6d3-1911adb57851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:52.023643 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a97f9-5883-45ea-a6d3-1911adb57851 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:52.023936 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a97f9-5883-45ea-a6d3-1911adb57851 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:52.024225 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a97f9-5883-45ea-a6d3-1911adb57851 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:52.024970 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a97f9-5883-45ea-a6d3-1911adb57851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:44:52.025348 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a97f9-5883-45ea-a6d3-1911adb57851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:52.026685 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a97f9-5883-45ea-a6d3-1911adb57851 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:52.041111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 602/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:44:52.045674 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:52.165228 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac8757-d313-4672-b9b0-3b007c0fd7f8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:52.202154 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663d677-504a-4aeb-b356-e9a746a1b262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:52.203079 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d677-504a-4aeb-b356-e9a746a1b262 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:52.203370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d677-504a-4aeb-b356-e9a746a1b262 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:52.203658 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d677-504a-4aeb-b356-e9a746a1b262 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:52.204472 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663d677-504a-4aeb-b356-e9a746a1b262 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:52.206074 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663d677-504a-4aeb-b356-e9a746a1b262 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:52.227797 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 603/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 9 15:44:52 2022] POST /identity/v3/users => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:44:54.796838 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0382c-f5a5-450e-8557-fea2d4f193d1 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:54.797439 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0382c-f5a5-450e-8557-fea2d4f193d1 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:54.797439 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0382c-f5a5-450e-8557-fea2d4f193d1 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:54.798943 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 603/1206] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:44:54.804361 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2678e7d-9719-48b2-8a72-01b3ef3611b8 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:54.804922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2678e7d-9719-48b2-8a72-01b3ef3611b8 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:54.804922 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2678e7d-9719-48b2-8a72-01b3ef3611b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:54.822595 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-c2678e7d-9719-48b2-8a72-01b3ef3611b8 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:44:54.823902 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-c2678e7d-9719-48b2-8a72-01b3ef3611b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:44:54.896438 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 604/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:44:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:44:54.925273 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0382c-f5a5-450e-8557-fea2d4f193d1 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:55.046508 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0382c-f5a5-450e-8557-fea2d4f193d1 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:55.090669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721499d-ae4d-4082-bbbc-0aed2286cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:55.091710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721499d-ae4d-4082-bbbc-0aed2286cbd5 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:55.092034 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721499d-ae4d-4082-bbbc-0aed2286cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:55.092367 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721499d-ae4d-4082-bbbc-0aed2286cbd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:55.105285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721499d-ae4d-4082-bbbc-0aed2286cbd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-f721499d-ae4d-4082-bbbc-0aed2286cbd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:44:55.114809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:44:55.116081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:44:55.117168 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 604/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:44:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:44:55.123652 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2678e7d-9719-48b2-8a72-01b3ef3611b8 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:55.239738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2678e7d-9719-48b2-8a72-01b3ef3611b8 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:55.278949 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:55.280320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:55.280613 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:55.280894 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:55.281695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:44:55.282017 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:55.283425 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:55.291810 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 605/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:44:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:44:55.298232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721499d-ae4d-4082-bbbc-0aed2286cbd5 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:55.409730 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721499d-ae4d-4082-bbbc-0aed2286cbd5 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:55.448527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:55.449379 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:55.449669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:55.450079 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:55.460758 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:55.462264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:44:55.480969 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:44:55.482303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:44:55.484098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:44:55.484098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:44:55.484098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:44:55.484098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:44:55.484098 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 605/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:44:55.489094 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:55.605518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1a855-0288-46e7-bf70-5f367a2cf967 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:55.642471 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:55.643256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:55.643697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:55.644042 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:55.644857 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:44:55.645251 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:55.646703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:55.663096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 606/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:44:55.668574 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:55.794628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136aaba-ffcb-4e60-b905-22cc4ad8ce92 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:55.835967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:55.836988 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:55.837472 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:55.837630 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] PATH_INFO: `/v3/users/placement` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:55.845918 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:55.847381 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:44:55.863289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:44:55.869384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:44:55.870865 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 606/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:44:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:44:55.874128 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:55.981238 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0301d2-5252-4f43-8a0b-5e26116ab02c None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:56.018110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:56.018799 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:56.019175 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:56.019365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:56.020213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:44:56.020528 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:56.021988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:56.047529 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 607/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:44:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:44:56.048761 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:56.149563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8d590-d2bc-4010-85fb-e83fef663d72 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:56.185493 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a96671-64a1-46ed-9722-1bd0adcadfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:56.186313 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a96671-64a1-46ed-9722-1bd0adcadfdc None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:56.186704 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a96671-64a1-46ed-9722-1bd0adcadfdc None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:56.187086 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a96671-64a1-46ed-9722-1bd0adcadfdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:56.257898 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a96671-64a1-46ed-9722-1bd0adcadfdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:56.259354 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a96671-64a1-46ed-9722-1bd0adcadfdc None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:56.364243 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 607/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:44:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:44:56.370068 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:56.472372 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a37e9c-40ea-46ae-930a-4baa0da9e6a7 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:56.511170 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:56.512160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:56.515933 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:56.516346 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:56.527256 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:56.528747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:44:56.550587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:44:56.552004 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:44:56.553308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:44:56.553308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:44:56.553308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:44:56.553308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:44:56.554326 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 608/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:44:56.560123 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a96671-64a1-46ed-9722-1bd0adcadfdc None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:56.668894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a96671-64a1-46ed-9722-1bd0adcadfdc None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:56.708182 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:56.708883 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:56.709209 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:56.709507 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:56.710239 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:44:56.710583 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:56.712083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:56.724605 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 608/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:44:56.730667 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:56.841486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b061d-6fe9-4186-bd50-42a7aa130b48 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:56.877681 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:56.878517 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:56.878771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:56.879109 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:56.885966 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:56.887417 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:44:56.902684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:44:56.903856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:44:56.905647 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 609/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:44:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:44:56.910329 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:57.020074 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ec89a-d7de-4b50-b2ee-3f2d56c2efa6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:57.058068 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af677cf0-f248-4fc5-bd45-2c7c7df66a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:57.058776 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af677cf0-f248-4fc5-bd45-2c7c7df66a9f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:57.059063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af677cf0-f248-4fc5-bd45-2c7c7df66a9f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:57.059380 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af677cf0-f248-4fc5-bd45-2c7c7df66a9f None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:57.060183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af677cf0-f248-4fc5-bd45-2c7c7df66a9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:44:57.061111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af677cf0-f248-4fc5-bd45-2c7c7df66a9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:57.062584 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af677cf0-f248-4fc5-bd45-2c7c7df66a9f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:57.078140 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 609/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:44:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:44:57.086643 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:57.193648 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70950986-567b-4bc3-a214-8cb006041ecd None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:57.229026 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318b88d-98ff-49a2-9c56-4e55bfb71831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:57.229753 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318b88d-98ff-49a2-9c56-4e55bfb71831 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:57.230049 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318b88d-98ff-49a2-9c56-4e55bfb71831 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:57.230330 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318b88d-98ff-49a2-9c56-4e55bfb71831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:57.231203 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318b88d-98ff-49a2-9c56-4e55bfb71831 None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=03ad505fa13b43ea85f25ec2e8bd3cd3) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:44:57.231524 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318b88d-98ff-49a2-9c56-4e55bfb71831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:57.233590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318b88d-98ff-49a2-9c56-4e55bfb71831 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:57.239963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 610/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:44:57 2022] GET /identity/v3/role_assignments?user.id=03ad505fa13b43ea85f25ec2e8bd3cd3&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:44:59.524311 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ebfc-de4a-40fd-bb0d-e6693ddaec68 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:59.524887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ebfc-de4a-40fd-bb0d-e6693ddaec68 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:59.524887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31ebfc-de4a-40fd-bb0d-e6693ddaec68 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:59.526417 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 610/1220] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:44:59.532834 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313e87f-c5e4-4334-9da4-8accb6af722c None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:59.533427 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313e87f-c5e4-4334-9da4-8accb6af722c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:59.533538 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313e87f-c5e4-4334-9da4-8accb6af722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:59.551290 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-7313e87f-c5e4-4334-9da4-8accb6af722c None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:44:59.552716 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-7313e87f-c5e4-4334-9da4-8accb6af722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:44:59.627074 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 611/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:44:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:44:59.650964 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31ebfc-de4a-40fd-bb0d-e6693ddaec68 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:59.758189 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31ebfc-de4a-40fd-bb0d-e6693ddaec68 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:59.791971 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899945e2-c430-40f3-a502-2a00359854fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:59.792717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899945e2-c430-40f3-a502-2a00359854fe None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:59.793004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899945e2-c430-40f3-a502-2a00359854fe None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:59.793308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899945e2-c430-40f3-a502-2a00359854fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:59.804142 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899945e2-c430-40f3-a502-2a00359854fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-899945e2-c430-40f3-a502-2a00359854fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:44:59.812727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:44:59.814172 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:44:59.815365 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:44:59.819031 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 611/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:44:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:44:59.820048 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313e87f-c5e4-4334-9da4-8accb6af722c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:44:59.936804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313e87f-c5e4-4334-9da4-8accb6af722c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:44:59.979453 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:44:59.981033 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:44:59.981356 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:44:59.981639 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:44:59.982975 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:44:59.983294 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:44:59.984768 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:44:59.994725 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 612/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:44:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:00.001386 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899945e2-c430-40f3-a502-2a00359854fe None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:00.108327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899945e2-c430-40f3-a502-2a00359854fe None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:00.149351 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:00.150299 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:00.150702 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:00.151081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:00.164175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:00.165908 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:00.184250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:00.186132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:00.187492 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:00.187492 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:00.187492 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:00.187492 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:00.190775 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 612/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:00.191842 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:00.303041 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebc2d4-1877-438d-a744-2cc2162565b0 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:00.341086 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:00.341817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:00.342319 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:00.342519 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:00.343410 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:00.343729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:00.345422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:00.359910 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 613/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:00.366012 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:00.470251 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8366ffb-ab2a-4cf6-ba72-283684775943 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:00.507412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:00.508241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:00.508508 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:00.508802 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:00.518318 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:00.519785 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:45:00.535006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:00.536711 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:00.537890 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 613/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:45:00.545484 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:00.647052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4111f02-0eaa-4851-a6c0-9e90f4309a60 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:00.682078 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:00.682820 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:00.683147 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:00.683429 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:00.684273 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:00.684628 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:00.686220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:00.708163 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 614/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:45:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:45:00.709467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:00.812456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdd0e6-57e9-41d8-81d5-f8e4780fe9f8 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:00.847592 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:00.848399 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:00.848690 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:00.848973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:00.859757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:00.861254 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:00.878451 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:00.879627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:00.880705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:00.880705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:00.880705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:00.880705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:00.880705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 614/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:00.885630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:00.988038 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf73421-46d3-446c-bfe6-fb984da4b5b8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:01.023438 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:01.024135 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:01.024427 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:01.024715 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:01.025500 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:01.025809 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:01.027193 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:01.039957 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 615/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:01.045817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:01.163493 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61aecb-3ffa-412e-b42d-e1e8bb01b00b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:01.208336 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:01.209567 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:01.209975 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:01.210258 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:01.218558 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:01.220309 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:45:01.235022 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:01.236195 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:01.243967 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 615/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:45:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:01.244673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:01.344465 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9ba26-b28c-4a4c-bef8-386a8e5f461e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:01.379352 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffc461-8872-4163-b1e1-e7c48bb02490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:01.380048 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffc461-8872-4163-b1e1-e7c48bb02490 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:01.380331 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffc461-8872-4163-b1e1-e7c48bb02490 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:01.380646 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffc461-8872-4163-b1e1-e7c48bb02490 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:01.381476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffc461-8872-4163-b1e1-e7c48bb02490 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:01.381869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffc461-8872-4163-b1e1-e7c48bb02490 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:01.383386 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffc461-8872-4163-b1e1-e7c48bb02490 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:01.396656 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 616/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:45:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:45:01.402951 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:01.521105 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f3fda-31eb-4877-9fad-735f9a0f7821 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:01.568568 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6438a1-c1fd-4a32-b597-1d55822d0888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:01.569561 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6438a1-c1fd-4a32-b597-1d55822d0888 None admin] REQUEST_METHOD: `PUT` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:01.569745 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6438a1-c1fd-4a32-b597-1d55822d0888 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:01.570017 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6438a1-c1fd-4a32-b597-1d55822d0888 None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/03ad505fa13b43ea85f25ec2e8bd3cd3/roles/d28cdc73a0fc42f6b6e0c45768d418ae` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:01.586271 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6438a1-c1fd-4a32-b597-1d55822d0888 None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=03ad505fa13b43ea85f25ec2e8bd3cd3, role_id=d28cdc73a0fc42f6b6e0c45768d418ae)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:01.587727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6438a1-c1fd-4a32-b597-1d55822d0888 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:01.600321 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 616/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:45:01 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/03ad505fa13b43ea85f25ec2e8bd3cd3/roles/d28cdc73a0fc42f6b6e0c45768d418ae => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 09 15:45:04.003645 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8fc70-cbe6-4794-87a3-0329128c2c98 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:04.004216 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8fc70-cbe6-4794-87a3-0329128c2c98 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:04.004216 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8fc70-cbe6-4794-87a3-0329128c2c98 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:04.005765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 617/1233] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:45:04.011760 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4decf6b-4819-403c-9cfd-dc1ae0b157d3 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:04.012232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4decf6b-4819-403c-9cfd-dc1ae0b157d3 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:04.012374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4decf6b-4819-403c-9cfd-dc1ae0b157d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:04.031577 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-b4decf6b-4819-403c-9cfd-dc1ae0b157d3 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:45:04.032796 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-b4decf6b-4819-403c-9cfd-dc1ae0b157d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:45:04.104925 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 617/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:45:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:45:04.129980 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8fc70-cbe6-4794-87a3-0329128c2c98 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:04.234656 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8fc70-cbe6-4794-87a3-0329128c2c98 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:04.272749 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e338e58-8e2a-49f2-8070-420bdce13452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:04.273605 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338e58-8e2a-49f2-8070-420bdce13452 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:04.273891 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338e58-8e2a-49f2-8070-420bdce13452 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:04.274173 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338e58-8e2a-49f2-8070-420bdce13452 None admin] PATH_INFO: `/v3/roles/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:04.284737 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e338e58-8e2a-49f2-8070-420bdce13452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5e338e58-8e2a-49f2-8070-420bdce13452 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:45:04.293723 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:04.295028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:04.296133 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 618/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 9 15:45:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:45:04.301846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4decf6b-4819-403c-9cfd-dc1ae0b157d3 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:04.403039 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4decf6b-4819-403c-9cfd-dc1ae0b157d3 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:04.436987 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:04.437688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:04.437978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:04.438260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:04.439025 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:04.439330 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:04.440777 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:04.450002 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 618/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 9 15:45:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:04.456424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e338e58-8e2a-49f2-8070-420bdce13452 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:04.562553 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e338e58-8e2a-49f2-8070-420bdce13452 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:04.601452 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:04.602258 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:04.602546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:04.602823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:04.614007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:04.615464 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:04.634533 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:04.635848 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:04.637081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:04.637081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:04.637081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:04.637081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:04.637081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 619/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:45:04.641746 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:04.750984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712be24-3b1a-4381-b406-b93567f6f6f0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:04.784554 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:04.785293 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:04.785586 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:04.785887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:04.786695 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:04.787009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:04.788384 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:04.800989 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 619/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:45:04.806917 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:04.922230 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6e238-a207-4fa9-a5bc-c7936754b328 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:04.968321 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:04.969542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:04.969831 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:04.970111 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] PATH_INFO: `/v3/users/placement` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:04.979273 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:04.980856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:45:04.996320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:05.002695 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:05.004724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 620/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:05.005626 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:05.112748 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8239a2a-a40c-4255-a76a-11a56642c09f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:05.159866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:05.160557 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:05.160852 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:05.161171 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:05.162034 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:05.162353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:05.163832 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:05.183824 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 620/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:45:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:45:05.190770 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:05.304885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503deca6-5fa9-4ea9-b73e-24cf99c0168f None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:05.343876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3b3c4-25d2-4a92-a0f8-bd7e8f30087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:05.344798 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3b3c4-25d2-4a92-a0f8-bd7e8f30087e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:05.345121 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3b3c4-25d2-4a92-a0f8-bd7e8f30087e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:05.345405 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3b3c4-25d2-4a92-a0f8-bd7e8f30087e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:05.415646 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3b3c4-25d2-4a92-a0f8-bd7e8f30087e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:05.417190 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3b3c4-25d2-4a92-a0f8-bd7e8f30087e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:05.528082 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 621/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:45:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:45:05.529823 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:05.646878 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddc0b7-df6c-457b-bc2c-4e69ed72197f None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:05.683851 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:05.684735 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:05.685026 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:05.685873 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:05.697358 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:05.699190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:05.720183 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:05.721304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:05.722691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:05.722691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:05.722691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:05.722691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:05.722691 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 621/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:45:05.727479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3b3c4-25d2-4a92-a0f8-bd7e8f30087e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:05.841997 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3b3c4-25d2-4a92-a0f8-bd7e8f30087e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:05.876931 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:05.877631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:05.877947 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:05.878202 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:05.879049 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:05.879368 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:05.880932 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:05.893060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 622/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:05.898812 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:06.013854 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c7c67-d7b9-470a-ab33-fadb5147439d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:06.050530 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:06.051338 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:06.051785 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:06.052095 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:06.059885 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:06.062064 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:45:06.076677 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:06.077984 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:06.079327 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 622/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:45:06.084078 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:06.190852 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cd914-b5d5-4630-a9fa-26beaa823ba0 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:06.225340 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca9510-29ea-4567-bc96-fc110852056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:06.226046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca9510-29ea-4567-bc96-fc110852056a None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:06.226332 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca9510-29ea-4567-bc96-fc110852056a None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:06.226675 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca9510-29ea-4567-bc96-fc110852056a None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:06.227522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca9510-29ea-4567-bc96-fc110852056a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:06.227822 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca9510-29ea-4567-bc96-fc110852056a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:06.229447 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca9510-29ea-4567-bc96-fc110852056a None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:06.243075 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 623/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:06.250218 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:06.362222 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52d691-467f-493f-a019-3c8ddbdb210b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:06.398495 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c595ab-3345-4298-b209-c47a64b2dad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:06.399157 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c595ab-3345-4298-b209-c47a64b2dad6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:06.399442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c595ab-3345-4298-b209-c47a64b2dad6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:06.399717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c595ab-3345-4298-b209-c47a64b2dad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:06.400552 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c595ab-3345-4298-b209-c47a64b2dad6 None admin] RBAC: Adding query filter params (role.id=d28cdc73a0fc42f6b6e0c45768d418ae, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=03ad505fa13b43ea85f25ec2e8bd3cd3) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:06.400866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c595ab-3345-4298-b209-c47a64b2dad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:06.402431 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c595ab-3345-4298-b209-c47a64b2dad6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:06.408703 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 623/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:45:06 2022] GET /identity/v3/role_assignments?user.id=03ad505fa13b43ea85f25ec2e8bd3cd3&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=d28cdc73a0fc42f6b6e0c45768d418ae => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:08.755096 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2000fff-fdbc-4ceb-af07-3cc5ed5b38f3 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:08.755754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2000fff-fdbc-4ceb-af07-3cc5ed5b38f3 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:08.755754 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2000fff-fdbc-4ceb-af07-3cc5ed5b38f3 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:08.757204 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 624/1247] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:45:08.762861 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d2b9c-172e-409c-a1e3-8b9f1bbecf68 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:08.763329 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d2b9c-172e-409c-a1e3-8b9f1bbecf68 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:08.763438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d2b9c-172e-409c-a1e3-8b9f1bbecf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:08.781262 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-2d3d2b9c-172e-409c-a1e3-8b9f1bbecf68 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:45:08.782566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-2d3d2b9c-172e-409c-a1e3-8b9f1bbecf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:45:08.859810 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 624/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:45:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:45:08.885744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2000fff-fdbc-4ceb-af07-3cc5ed5b38f3 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:09.003237 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2000fff-fdbc-4ceb-af07-3cc5ed5b38f3 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:09.044783 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17907fbf-3195-4b04-b76f-a2f5a1896652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:09.045730 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17907fbf-3195-4b04-b76f-a2f5a1896652 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:09.046065 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17907fbf-3195-4b04-b76f-a2f5a1896652 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:09.046410 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17907fbf-3195-4b04-b76f-a2f5a1896652 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:09.058759 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17907fbf-3195-4b04-b76f-a2f5a1896652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-17907fbf-3195-4b04-b76f-a2f5a1896652 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:45:09.067627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:09.068927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:09.070098 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 625/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:45:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:09.075409 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d2b9c-172e-409c-a1e3-8b9f1bbecf68 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:09.182582 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d2b9c-172e-409c-a1e3-8b9f1bbecf68 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:09.218236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:09.219180 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:09.219466 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:09.219757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:09.220519 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:09.220833 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:09.222255 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:09.230491 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 625/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:45:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:09.236181 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17907fbf-3195-4b04-b76f-a2f5a1896652 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:09.340642 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17907fbf-3195-4b04-b76f-a2f5a1896652 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:09.378967 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:09.379819 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:09.380100 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:09.380421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:09.393788 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:09.395249 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:09.415505 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:09.416855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:09.418194 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:09.418194 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:09.418194 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:09.418194 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:09.418194 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 626/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:09.422930 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:09.547415 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d86584-14b5-47b2-943e-1936bbf4252a None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:09.589821 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:09.590627 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:09.591073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:09.591432 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:09.592456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:09.592775 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:09.594731 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:09.611175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 626/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:09.616844 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:09.735597 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fab451-6333-41af-9440-03a78307f4f6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:09.771978 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:09.772849 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:09.773178 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:09.773468 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] PATH_INFO: `/v3/users/placement` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:09.784062 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:09.785738 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:45:09.802877 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:09.804313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:09.805518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 627/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:09.810869 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:09.915220 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36da2f5-7e8a-4e13-aed3-916475a1ada9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:09.950872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:09.951600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:09.951949 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:09.952224 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:09.953173 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:09.953479 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:09.955063 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:09.974922 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 627/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:45:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 09 15:45:09.980703 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:10.096745 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd2784-ed46-4b1f-a1e9-412d9756550d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:10.132963 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726b0e7-cff4-421a-8993-f4400e761f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:10.133649 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726b0e7-cff4-421a-8993-f4400e761f51 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:10.133938 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726b0e7-cff4-421a-8993-f4400e761f51 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:10.134242 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726b0e7-cff4-421a-8993-f4400e761f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:10.200968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726b0e7-cff4-421a-8993-f4400e761f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:10.202459 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726b0e7-cff4-421a-8993-f4400e761f51 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:10.310122 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 628/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:45:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:45:10.316138 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:10.422660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bce52-f82a-4b9b-8955-1933719f6cf6 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:10.459973 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:10.460819 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:10.461141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:10.461458 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:10.473640 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:10.475108 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:10.493494 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:10.494709 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:10.495870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:10.495870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:10.495870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:10.495870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:10.495870 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 628/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:45:10.500497 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726b0e7-cff4-421a-8993-f4400e761f51 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:10.611771 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726b0e7-cff4-421a-8993-f4400e761f51 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:10.651419 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:10.652099 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:10.652387 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:10.652666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:10.653424 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:10.653759 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:10.655656 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:10.668253 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 629/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:10.674175 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:10.778637 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca70f5-6ee5-45ba-8d91-1dfe64de59c0 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:10.812806 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:10.813652 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:10.813939 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:10.814285 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:10.821029 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:10.822488 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:45:10.838398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:10.839817 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:10.841149 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 629/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:10.846988 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:10.954581 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27418-6d47-4481-91c1-c999f196f8bb None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:10.991407 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103575f-aed3-4541-a24f-87279c263d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:10.992077 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103575f-aed3-4541-a24f-87279c263d27 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:10.992408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103575f-aed3-4541-a24f-87279c263d27 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:10.992684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103575f-aed3-4541-a24f-87279c263d27 None admin] PATH_INFO: `/v3/projects` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:10.993582 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103575f-aed3-4541-a24f-87279c263d27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:10.993899 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103575f-aed3-4541-a24f-87279c263d27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:10.995664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103575f-aed3-4541-a24f-87279c263d27 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:11.008986 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 630/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:11.016080 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:11.131181 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462da2c-e22b-4b0c-b39f-702e13cd4a0e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:11.171376 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1608c5b-515f-4fd9-926e-1d9b79f490c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:11.172215 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1608c5b-515f-4fd9-926e-1d9b79f490c1 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:11.172632 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1608c5b-515f-4fd9-926e-1d9b79f490c1 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:11.172960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1608c5b-515f-4fd9-926e-1d9b79f490c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:11.174076 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1608c5b-515f-4fd9-926e-1d9b79f490c1 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=03ad505fa13b43ea85f25ec2e8bd3cd3) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:11.174578 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1608c5b-515f-4fd9-926e-1d9b79f490c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:11.176143 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1608c5b-515f-4fd9-926e-1d9b79f490c1 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:11.183596 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 630/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:45:11 2022] GET /identity/v3/role_assignments?user.id=03ad505fa13b43ea85f25ec2e8bd3cd3&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=ebff922955b74c919d58613875326151 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:13.416244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763b61-3033-4ac0-ad0e-8e4a18502daf None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:13.416794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763b61-3033-4ac0-ad0e-8e4a18502daf None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:13.416794 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763b61-3033-4ac0-ad0e-8e4a18502daf None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:13.418234 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 631/1261] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:45:13.423527 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb120b-1c37-4f16-8750-d7bb4e4439bf None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:13.424007 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb120b-1c37-4f16-8750-d7bb4e4439bf None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:13.424118 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb120b-1c37-4f16-8750-d7bb4e4439bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:13.441127 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-15cb120b-1c37-4f16-8750-d7bb4e4439bf None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:45:13.442315 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-15cb120b-1c37-4f16-8750-d7bb4e4439bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:45:13.505703 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 631/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:45:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 09 15:45:13.527936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763b61-3033-4ac0-ad0e-8e4a18502daf None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:13.648791 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763b61-3033-4ac0-ad0e-8e4a18502daf None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:13.682099 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f5bb2-ff3c-4bd4-909e-d2f0ef47a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:13.682829 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f5bb2-ff3c-4bd4-909e-d2f0ef47a67e None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:13.683118 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f5bb2-ff3c-4bd4-909e-d2f0ef47a67e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:13.683401 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f5bb2-ff3c-4bd4-909e-d2f0ef47a67e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:13.693684 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f5bb2-ff3c-4bd4-909e-d2f0ef47a67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-ee8f5bb2-ff3c-4bd4-909e-d2f0ef47a67e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:45:13.701364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:13.702627 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:13.703805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 632/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:45:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:13.708585 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb120b-1c37-4f16-8750-d7bb4e4439bf None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:13.803159 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb120b-1c37-4f16-8750-d7bb4e4439bf None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:13.835304 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:13.835961 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:13.836251 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:13.836532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] PATH_INFO: `/v3/roles` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:13.837310 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] RBAC: Adding query filter params (name=admin) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:13.837624 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:13.838979 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:13.846894 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 632/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:45:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:13.852549 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f5bb2-ff3c-4bd4-909e-d2f0ef47a67e None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:13.949960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f5bb2-ff3c-4bd4-909e-d2f0ef47a67e None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:13.982912 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:13.983708 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:13.983999 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:13.984284 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:13.994489 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:13.995952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:14.012546 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:14.013757 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:14.015189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:14.015189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:14.015189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:14.015189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:14.015189 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 633/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:14.019514 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:14.114750 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cffab-bdd5-4604-bea3-aee87c81c65e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:14.147066 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:14.147717 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:14.148004 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:14.148283 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:14.149007 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:14.149353 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:14.150725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:14.162564 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 633/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:14.168110 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:14.263951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54affb63-5d36-4eca-8ff1-8392ed70df45 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:14.295874 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:14.296664 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:14.296956 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:14.297272 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:14.304432 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:14.305901 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:45:14.320555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:14.321855 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:14.323215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 634/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:14.327406 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:14.422361 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c79662-1199-473f-9a5b-321fcbb14163 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:14.454643 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:14.455308 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:14.455597 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:14.455879 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] PATH_INFO: `/v3/users` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:14.456723 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:14.457067 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:14.458503 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:14.474988 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 634/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:45:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:14.480553 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:14.585869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f6e41-056c-422b-abd4-c0ec64b3fbd6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:14.618697 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:14.619494 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:14.619785 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:14.620069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:14.630841 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:14.632295 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:14.648825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:14.650080 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:14.651367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:14.651367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:14.651367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:14.651367 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:14.653204 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 635/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:14.656314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:14.751983 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d1f98-2fcc-4530-9cc4-d276d67a8a0b None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:14.785025 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:14.785714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:14.786006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:14.786290 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:14.787019 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:14.787335 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:14.788715 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:14.800184 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 635/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:14.805758 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:14.901499 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7bdf10-1bff-4047-bb88-c6e63f88347b None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:14.935408 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:14.936226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:14.936519 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:14.936807 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:14.943501 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:14.944946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:45:14.959823 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:14.961028 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:14.962317 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 636/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:14.967423 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:15.064461 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6dd9c-f66a-4508-a465-ee2c6703deeb None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:15.096747 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe6f88-c3e1-4fa4-a27d-c9963c2393d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:15.097433 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6f88-c3e1-4fa4-a27d-c9963c2393d9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:15.097727 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6f88-c3e1-4fa4-a27d-c9963c2393d9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:15.098016 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6f88-c3e1-4fa4-a27d-c9963c2393d9 None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:15.098814 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe6f88-c3e1-4fa4-a27d-c9963c2393d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:15.099132 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe6f88-c3e1-4fa4-a27d-c9963c2393d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:15.100563 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe6f88-c3e1-4fa4-a27d-c9963c2393d9 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:15.112705 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 636/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:45:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:15.118525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:15.218005 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a6b18-61b2-4d2a-a821-f91672ba8974 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:15.250996 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9464a8-31fe-44f6-be16-bf784c01623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:15.251875 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9464a8-31fe-44f6-be16-bf784c01623e None admin] REQUEST_METHOD: `PUT` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:15.252160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9464a8-31fe-44f6-be16-bf784c01623e None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:15.252440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9464a8-31fe-44f6-be16-bf784c01623e None admin] PATH_INFO: `/v3/projects/290042e8fb734d94baf915970d7d7b95/users/03ad505fa13b43ea85f25ec2e8bd3cd3/roles/ebff922955b74c919d58613875326151` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:15.254412 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9464a8-31fe-44f6-be16-bf784c01623e None admin] RBAC: Authorizing `identity:create_grant(project_id=290042e8fb734d94baf915970d7d7b95, user_id=03ad505fa13b43ea85f25ec2e8bd3cd3, role_id=ebff922955b74c919d58613875326151)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:15.255839 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9464a8-31fe-44f6-be16-bf784c01623e None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:15.268343 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 637/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 9 15:45:15 2022] PUT /identity/v3/projects/290042e8fb734d94baf915970d7d7b95/users/03ad505fa13b43ea85f25ec2e8bd3cd3/roles/ebff922955b74c919d58613875326151 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 09 15:45:17.499658 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242147a-7847-4549-af61-799e432d5ad1 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:17.500232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242147a-7847-4549-af61-799e432d5ad1 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:17.500232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242147a-7847-4549-af61-799e432d5ad1 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:17.501620 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 637/1274] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:45:17.506744 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd1a16-a3a5-403b-9630-1558b0979e74 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:17.507200 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd1a16-a3a5-403b-9630-1558b0979e74 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:17.507314 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd1a16-a3a5-403b-9630-1558b0979e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:17.524231 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-fefd1a16-a3a5-403b-9630-1558b0979e74 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:45:17.525456 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-fefd1a16-a3a5-403b-9630-1558b0979e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:45:17.602299 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 638/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:45:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:45:17.626573 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242147a-7847-4549-af61-799e432d5ad1 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:17.721901 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242147a-7847-4549-af61-799e432d5ad1 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:17.753929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5896-4be6-4c79-b95d-976c53fc6df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:17.754645 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5896-4be6-4c79-b95d-976c53fc6df9 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:17.754929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5896-4be6-4c79-b95d-976c53fc6df9 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:17.755206 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5896-4be6-4c79-b95d-976c53fc6df9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:17.765475 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f5896-4be6-4c79-b95d-976c53fc6df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-e58f5896-4be6-4c79-b95d-976c53fc6df9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(role_id) May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 09 15:45:17.772920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:17.774210 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:17.775625 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 638/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 9 15:45:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:17.780011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd1a16-a3a5-403b-9630-1558b0979e74 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:17.872944 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd1a16-a3a5-403b-9630-1558b0979e74 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:17.904673 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:17.905365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:17.905650 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:17.905929 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] PATH_INFO: `/v3/roles` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:17.906682 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:17.906990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:17.908349 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:17.916421 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 639/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 9 15:45:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:17.922024 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5896-4be6-4c79-b95d-976c53fc6df9 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:18.018112 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5896-4be6-4c79-b95d-976c53fc6df9 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:18.049623 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:18.050390 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:18.050676 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:18.050953 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:18.061459 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:18.062887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:18.079096 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:18.080289 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:18.081606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:18.081606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:18.081606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:18.081606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:18.081606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 639/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:18.085990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:18.181059 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758d73c-6b7a-4494-8846-4c791c9a09e7 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:18.212630 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:18.213314 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:18.213606 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:18.213887 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] PATH_INFO: `/v3/domains` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:18.214614 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:18.214925 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:18.216587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:18.228095 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 640/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:18.234354 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:18.328959 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4904b-eec8-4b47-9fc5-8940ffb1f385 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:18.360872 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:18.361690 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:18.361981 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:18.362260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] PATH_INFO: `/v3/users/placement` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:18.369504 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:18.370937 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_user(user_id) May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 09 15:45:18.385438 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:18.386678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:18.387948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 640/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:18.392280 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:18.487148 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2648f6-9f40-4bef-bc8f-1f2253bcce5d None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:18.518935 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:18.519602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:18.519892 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:18.520176 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] PATH_INFO: `/v3/users` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:18.521018 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:18.521365 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:18.522795 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:18.540900 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 641/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 9 15:45:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:18.547952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:18.644864 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd6636-d574-45e5-834a-dd49c1854fcd None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:18.677434 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c0d0e-9662-4eb1-bf26-82d69c01ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:18.678084 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c0d0e-9662-4eb1-bf26-82d69c01ac34 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:18.678374 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c0d0e-9662-4eb1-bf26-82d69c01ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:18.678656 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c0d0e-9662-4eb1-bf26-82d69c01ac34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:18.742083 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c0d0e-9662-4eb1-bf26-82d69c01ac34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:18.743529 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c0d0e-9662-4eb1-bf26-82d69c01ac34 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:18.836866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 641/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 9 15:45:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 09 15:45:18.843274 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:18.936220 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcb529-4a62-467f-ae71-fc5aa2d844a8 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:18.968463 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:18.969528 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:18.969814 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:18.970089 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:18.979590 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:18.981019 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 09 15:45:18.997720 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:18.998884 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:19.000011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 09 15:45:19.000011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 09 15:45:19.000011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 09 15:45:19.000011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:19.000011 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 642/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 9 15:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:19.004546 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c0d0e-9662-4eb1-bf26-82d69c01ac34 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:19.099493 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c0d0e-9662-4eb1-bf26-82d69c01ac34 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:19.131611 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:19.132269 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:19.132560 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:19.132845 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] PATH_INFO: `/v3/domains` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:19.133613 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:19.133929 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:19.135310 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:19.146716 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 642/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 9 15:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:19.152369 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:19.250313 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c4223-6092-4fb4-a1b0-502440e71cde None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:19.282681 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:19.283473 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:19.283767 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:19.284054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:19.290555 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:19.292002 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(project_id) May 09 15:45:19.305573 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application with Lock( May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:19.306856 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: ERROR keystone.server.flask.application May 09 15:45:19.308278 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 643/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 9 15:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:19.313070 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:19.407360 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76b1de-4081-4feb-aee2-2a3a968a6a7d None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:19.438679 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06929de6-5924-4d05-99b7-b3f54c12a17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:19.439317 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929de6-5924-4d05-99b7-b3f54c12a17b None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:19.439600 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929de6-5924-4d05-99b7-b3f54c12a17b None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:19.439876 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06929de6-5924-4d05-99b7-b3f54c12a17b None admin] PATH_INFO: `/v3/projects` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:19.440660 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06929de6-5924-4d05-99b7-b3f54c12a17b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:19.440972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06929de6-5924-4d05-99b7-b3f54c12a17b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:19.442410 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06929de6-5924-4d05-99b7-b3f54c12a17b None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:19.454260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 643/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 9 15:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:19.460066 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:19.559083 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791c1f6-a06a-4d3c-bd02-011d8d228ce7 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:19.591617 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f2fbe-bb3e-4e97-8bd0-65e5d23e16d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:19.592277 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2fbe-bb3e-4e97-8bd0-65e5d23e16d7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:19.592571 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2fbe-bb3e-4e97-8bd0-65e5d23e16d7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:19.592859 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2fbe-bb3e-4e97-8bd0-65e5d23e16d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:19.594055 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f2fbe-bb3e-4e97-8bd0-65e5d23e16d7 None admin] RBAC: Adding query filter params (role.id=ebff922955b74c919d58613875326151, scope.project.id=290042e8fb734d94baf915970d7d7b95, user.id=03ad505fa13b43ea85f25ec2e8bd3cd3) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:19.594384 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f2fbe-bb3e-4e97-8bd0-65e5d23e16d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:19.595815 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f2fbe-bb3e-4e97-8bd0-65e5d23e16d7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:19.601552 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 644/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 9 15:45:19 2022] GET /identity/v3/role_assignments?user.id=03ad505fa13b43ea85f25ec2e8bd3cd3&scope.project.id=290042e8fb734d94baf915970d7d7b95&role.id=ebff922955b74c919d58613875326151 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:21.829548 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615abbc-4505-49fc-92f6-767515e86448 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:21.830111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615abbc-4505-49fc-92f6-767515e86448 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:21.830111 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615abbc-4505-49fc-92f6-767515e86448 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:21.831467 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 644/1288] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:45:21.836663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5364-28e5-43c6-b429-14739182b816 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:21.837139 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5364-28e5-43c6-b429-14739182b816 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:21.837271 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5364-28e5-43c6-b429-14739182b816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:21.853969 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-3f9d5364-28e5-43c6-b429-14739182b816 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:45:21.855168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-3f9d5364-28e5-43c6-b429-14739182b816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:45:21.918538 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 645/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:45:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:45:21.941765 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615abbc-4505-49fc-92f6-767515e86448 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:22.042920 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615abbc-4505-49fc-92f6-767515e86448 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:22.076126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:22.076923 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:22.077247 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:22.077532 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] PATH_INFO: `/v3/services/placement` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:22.078192 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:22.079619 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:22.091141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:22.092541 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:22.093628 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 645/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:45:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:22.098783 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d5364-28e5-43c6-b429-14739182b816 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:22.192329 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d5364-28e5-43c6-b429-14739182b816 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:22.224525 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfbb38-2c77-479d-a9cc-5bae53f397b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:22.225227 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfbb38-2c77-479d-a9cc-5bae53f397b8 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:22.225522 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfbb38-2c77-479d-a9cc-5bae53f397b8 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:22.225803 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfbb38-2c77-479d-a9cc-5bae53f397b8 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:22.226516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfbb38-2c77-479d-a9cc-5bae53f397b8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:22.226831 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfbb38-2c77-479d-a9cc-5bae53f397b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:22.228197 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfbb38-2c77-479d-a9cc-5bae53f397b8 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:22.233476 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 646/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:45:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:22.239053 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:22.338188 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85ff0c-abda-4a03-b153-00b81c3ce110 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:22.370692 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718f805-0b8b-4b0a-90b0-e2661004347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:22.371350 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718f805-0b8b-4b0a-90b0-e2661004347c None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:22.371637 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718f805-0b8b-4b0a-90b0-e2661004347c None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:22.371915 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718f805-0b8b-4b0a-90b0-e2661004347c None admin] PATH_INFO: `/v3/services` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:22.372619 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718f805-0b8b-4b0a-90b0-e2661004347c None admin] RBAC: Adding query filter params (type=placement) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:22.372934 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718f805-0b8b-4b0a-90b0-e2661004347c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:22.374314 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718f805-0b8b-4b0a-90b0-e2661004347c None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:22.378540 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 646/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:45:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:24.606920 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de341-d593-4d19-9170-fa95c1ce3fe0 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:24.607477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de341-d593-4d19-9170-fa95c1ce3fe0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:24.607477 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140de341-d593-4d19-9170-fa95c1ce3fe0 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:24.608867 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 647/1293] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:45:24.614054 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcc165-2818-4423-8157-3e9f4898979d None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:24.614473 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcc165-2818-4423-8157-3e9f4898979d None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:24.614617 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcc165-2818-4423-8157-3e9f4898979d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:24.631369 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-76bcc165-2818-4423-8157-3e9f4898979d None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:45:24.632565 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-76bcc165-2818-4423-8157-3e9f4898979d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:45:24.694723 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 647/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:45:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:45:24.717553 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140de341-d593-4d19-9170-fa95c1ce3fe0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:24.816309 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140de341-d593-4d19-9170-fa95c1ce3fe0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:24.848226 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2a64d-8dc6-4607-bced-1e1e10b19254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:24.848890 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a64d-8dc6-4607-bced-1e1e10b19254 None admin] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:24.849217 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a64d-8dc6-4607-bced-1e1e10b19254 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:24.849524 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a64d-8dc6-4607-bced-1e1e10b19254 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:24.850236 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2a64d-8dc6-4607-bced-1e1e10b19254 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:24.851603 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2a64d-8dc6-4607-bced-1e1e10b19254 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:24.864572 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 648/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 9 15:45:24 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:45:27.096253 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b13991-5266-4884-80cd-4a61d5cc9920 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:27.096793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b13991-5266-4884-80cd-4a61d5cc9920 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:27.096793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b13991-5266-4884-80cd-4a61d5cc9920 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:27.098263 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 648/1296] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:45:27.103672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57066-bf7e-49db-90cc-32b47a5d262b None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:27.104093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57066-bf7e-49db-90cc-32b47a5d262b None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:27.104241 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57066-bf7e-49db-90cc-32b47a5d262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:27.120520 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-b4a57066-bf7e-49db-90cc-32b47a5d262b None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:45:27.121750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-b4a57066-bf7e-49db-90cc-32b47a5d262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:45:27.191114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 649/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:45:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:45:27.214106 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b13991-5266-4884-80cd-4a61d5cc9920 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:27.311566 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b13991-5266-4884-80cd-4a61d5cc9920 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:27.344455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:27.345265 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:27.345554 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:27.345833 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] PATH_INFO: `/v3/services/placement` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:27.346489 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:27.347844 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:27.359352 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:27.360700 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:27.361974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 649/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:45:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 09 15:45:27.366729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a57066-bf7e-49db-90cc-32b47a5d262b None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:27.460329 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a57066-bf7e-49db-90cc-32b47a5d262b None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:27.492494 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:27.493193 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:27.493487 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:27.493770 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:27.494486 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:27.494802 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:27.496173 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:27.501770 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 650/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:45:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:27.507616 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:27.604124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a465354-5bb2-4068-95df-1476f8e91b89 None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:27.635522 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8526c5-bdd3-4f44-8442-bfd40eb59459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:27.636165 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8526c5-bdd3-4f44-8442-bfd40eb59459 None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:27.636454 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8526c5-bdd3-4f44-8442-bfd40eb59459 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:27.636732 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8526c5-bdd3-4f44-8442-bfd40eb59459 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:27.637551 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8526c5-bdd3-4f44-8442-bfd40eb59459 None admin] RBAC: Adding query filter params (interface=public, service_id=329b2b45fbaa4f9ea63632758cf5e712, region_id=RegionOne) {{(pid=275712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:27.638426 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8526c5-bdd3-4f44-8442-bfd40eb59459 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:27.639866 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8526c5-bdd3-4f44-8442-bfd40eb59459 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:27.644861 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 650/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 9 15:45:27 2022] GET /identity/v3/endpoints?service_id=329b2b45fbaa4f9ea63632758cf5e712&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:27.650574 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:27.744335 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbbb35-35c6-4b5b-8bc9-8d00a1b757b6 None admin] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:27.776433 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b15f6-ede3-4cc2-b4ef-71ce35cfd1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:27.777112 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b15f6-ede3-4cc2-b4ef-71ce35cfd1f3 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:27.777399 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b15f6-ede3-4cc2-b4ef-71ce35cfd1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:27.777679 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b15f6-ede3-4cc2-b4ef-71ce35cfd1f3 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:27.778396 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b15f6-ede3-4cc2-b4ef-71ce35cfd1f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:27.779750 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b15f6-ede3-4cc2-b4ef-71ce35cfd1f3 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:27.786304 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 651/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 9 15:45:27 2022] GET /identity/v3/services => generated 2191 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 09 15:45:29.969290 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553c35-675b-4738-b811-0326d514d234 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:29.969796 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553c35-675b-4738-b811-0326d514d234 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:29.969902 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553c35-675b-4738-b811-0326d514d234 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:29.971213 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 651/1302] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:45:29.976458 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e029c7-53fa-444b-b4ef-f35b5de54bfe None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:29.976899 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e029c7-53fa-444b-b4ef-f35b5de54bfe None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:29.977009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e029c7-53fa-444b-b4ef-f35b5de54bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:29.993935 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-07e029c7-53fa-444b-b4ef-f35b5de54bfe None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:45:29.995124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-07e029c7-53fa-444b-b4ef-f35b5de54bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:45:30.057495 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 652/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:45:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 09 15:45:30.080189 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47553c35-675b-4738-b811-0326d514d234 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:30.176447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47553c35-675b-4738-b811-0326d514d234 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:30.207850 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:30.208664 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:30.208952 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:30.209260 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] PATH_INFO: `/v3/services/placement` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:30.209917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:30.211274 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application Traceback (most recent call last): May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 09 15:45:30.223678 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._get_service(service_id) May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 09 15:45:30.225113 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.get_or_create( May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application with Lock( May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self._enter() May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application return self.creator() May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application created_value = creator( May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: ERROR keystone.server.flask.application May 09 15:45:30.226404 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 652/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 9 15:45:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 09 15:45:30.230773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e029c7-53fa-444b-b4ef-f35b5de54bfe None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:30.329805 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e029c7-53fa-444b-b4ef-f35b5de54bfe None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:30.362273 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec2782-edbf-4626-a494-ce2ed2dcd1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:30.362937 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2782-edbf-4626-a494-ce2ed2dcd1c7 None admin] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:30.363225 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2782-edbf-4626-a494-ce2ed2dcd1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:30.363507 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2782-edbf-4626-a494-ce2ed2dcd1c7 None admin] PATH_INFO: `/v3/services` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:30.364225 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec2782-edbf-4626-a494-ce2ed2dcd1c7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=275711) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 09 15:45:30.364541 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec2782-edbf-4626-a494-ce2ed2dcd1c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:30.365936 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec2782-edbf-4626-a494-ce2ed2dcd1c7 None admin] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:30.370860 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 653/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 9 15:45:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 09 15:45:30.377126 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:45:30.470816 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e34908-dfac-4c7f-b509-21f68c78728e None admin] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:45:30.502031 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a7471-99d7-4aa0-ba0b-9313892a2d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2a1baaf61604444b2c20a19956c1591', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:45:30.502673 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7471-99d7-4aa0-ba0b-9313892a2d62 None admin] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:45:30.502960 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7471-99d7-4aa0-ba0b-9313892a2d62 None admin] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:45:30.503239 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7471-99d7-4aa0-ba0b-9313892a2d62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:45:30.503974 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a7471-99d7-4aa0-ba0b-9313892a2d62 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:45:30.505351 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a7471-99d7-4aa0-ba0b-9313892a2d62 None admin] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:45:30.522948 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 653/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 9 15:45:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 09 15:46:36.044880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393027c7-fa12-4106-bf41-8168fd18052c None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:36.045493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393027c7-fa12-4106-bf41-8168fd18052c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:36.045493 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393027c7-fa12-4106-bf41-8168fd18052c None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:36.046746 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 654/1307] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:36.052294 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f6f6-f978-43c5-8883-28491460dc98 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:36.052782 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f6f6-f978-43c5-8883-28491460dc98 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:36.053122 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909f6f6-f978-43c5-8883-28491460dc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:36.220323 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-a909f6f6-f978-43c5-8883-28491460dc98 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:36.221775 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-a909f6f6-f978-43c5-8883-28491460dc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:36.449913 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 654/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:36.516315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f376b-85bd-4044-b75c-65bdb1880b97 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:36.516602 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f376b-85bd-4044-b75c-65bdb1880b97 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:36.517124 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f376b-85bd-4044-b75c-65bdb1880b97 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:36.518701 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 655/1309] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 9 15:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:36.524978 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5fb52-f2a7-4ab2-846e-4a99b468ed8e None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:36.525442 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5fb52-f2a7-4ab2-846e-4a99b468ed8e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:36.525836 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5fb52-f2a7-4ab2-846e-4a99b468ed8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:36.680044 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-74b5fb52-f2a7-4ab2-846e-4a99b468ed8e None None] MFA Rules not processed for user `88be715120ee4267a4a206b71fa21224`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:36.681307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-74b5fb52-f2a7-4ab2-846e-4a99b468ed8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:36.851614 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 655/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 9 15:46:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:36.871826 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f376b-85bd-4044-b75c-65bdb1880b97 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:46:36.875724 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-174f376b-85bd-4044-b75c-65bdb1880b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:37.031173 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f376b-85bd-4044-b75c-65bdb1880b97 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:46:37.109392 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acda3ab-75c9-4fa1-8536-588846c5c8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:46:37.110885 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acda3ab-75c9-4fa1-8536-588846c5c8ee service neutron] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:37.111375 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acda3ab-75c9-4fa1-8536-588846c5c8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:37.111868 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acda3ab-75c9-4fa1-8536-588846c5c8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:37.113747 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-3acda3ab-75c9-4fa1-8536-588846c5c8ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:37.227026 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acda3ab-75c9-4fa1-8536-588846c5c8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:46:37.228696 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acda3ab-75c9-4fa1-8536-588846c5c8ee service neutron] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:46:37.249314 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 656/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:46:39.895319 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52221caf-6f4a-4e34-82e4-8293f0acad80 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:39.896331 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52221caf-6f4a-4e34-82e4-8293f0acad80 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:39.896869 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52221caf-6f4a-4e34-82e4-8293f0acad80 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:39.905206 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedcddd-15c1-487a-90a1-f65ffc91bd07 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:39.905206 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedcddd-15c1-487a-90a1-f65ffc91bd07 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:39.905206 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedcddd-15c1-487a-90a1-f65ffc91bd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:39.909439 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 656/1312] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:46:39 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:39.986370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-bdedcddd-15c1-487a-90a1-f65ffc91bd07 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:39.988331 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-bdedcddd-15c1-487a-90a1-f65ffc91bd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:40.093165 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 657/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:46:40.148986 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad78ce-7b00-49f6-b8c5-60eed65771bb None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:40.149618 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad78ce-7b00-49f6-b8c5-60eed65771bb None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:40.150278 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad78ce-7b00-49f6-b8c5-60eed65771bb None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:40.152340 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 657/1314] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 9 15:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:40.159239 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150da61-f2b1-4325-b75a-a3b9031c0d6b None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:40.159544 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150da61-f2b1-4325-b75a-a3b9031c0d6b None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:40.159946 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150da61-f2b1-4325-b75a-a3b9031c0d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:40.236787 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-6150da61-f2b1-4325-b75a-a3b9031c0d6b None None] MFA Rules not processed for user `88be715120ee4267a4a206b71fa21224`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:40.238053 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-6150da61-f2b1-4325-b75a-a3b9031c0d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:40.321990 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 658/1315] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 9 15:46:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:40.332609 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad78ce-7b00-49f6-b8c5-60eed65771bb None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:46:40.334766 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-c4ad78ce-7b00-49f6-b8c5-60eed65771bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:40.457190 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad78ce-7b00-49f6-b8c5-60eed65771bb None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:46:40.546038 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11e66f-cb5d-44e1-ba94-37e3f2def74b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:46:40.547123 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e66f-cb5d-44e1-ba94-37e3f2def74b service neutron] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:40.547657 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e66f-cb5d-44e1-ba94-37e3f2def74b service neutron] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:40.548186 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e66f-cb5d-44e1-ba94-37e3f2def74b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:40.550882 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-ef11e66f-cb5d-44e1-ba94-37e3f2def74b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:40.689716 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11e66f-cb5d-44e1-ba94-37e3f2def74b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:46:40.693545 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11e66f-cb5d-44e1-ba94-37e3f2def74b service neutron] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:46:40.745026 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 658/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:46:43.527370 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6199f82-73bc-4aac-88f8-b5c6769a9816 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:43.527951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6199f82-73bc-4aac-88f8-b5c6769a9816 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:43.527951 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6199f82-73bc-4aac-88f8-b5c6769a9816 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:43.529428 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 659/1317] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:46:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:43.534669 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb552e95-e28b-4d2f-9b89-d95258677268 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:43.535065 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb552e95-e28b-4d2f-9b89-d95258677268 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:43.535242 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb552e95-e28b-4d2f-9b89-d95258677268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:43.600846 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-cb552e95-e28b-4d2f-9b89-d95258677268 None None] MFA Rules not processed for user `2a6b5024341c429e891c0f92594e0c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:43.602105 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-cb552e95-e28b-4d2f-9b89-d95258677268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:43.695068 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 659/1318] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:46:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:43.738060 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6199f82-73bc-4aac-88f8-b5c6769a9816 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:46:43.739678 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-d6199f82-73bc-4aac-88f8-b5c6769a9816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:43.845825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6199f82-73bc-4aac-88f8-b5c6769a9816 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:46:43.917894 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620976cd-fb46-41e1-ae1f-f2bc960b5bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:46:43.918052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620976cd-fb46-41e1-ae1f-f2bc960b5bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:43.918213 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620976cd-fb46-41e1-ae1f-f2bc960b5bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:43.918495 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620976cd-fb46-41e1-ae1f-f2bc960b5bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:43.920156 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-620976cd-fb46-41e1-ae1f-f2bc960b5bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:44.032232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620976cd-fb46-41e1-ae1f-f2bc960b5bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:46:44.033735 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620976cd-fb46-41e1-ae1f-f2bc960b5bf8 service neutron] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:46:44.051739 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 660/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:46:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:46:46.849500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e317fbd-7398-4ecd-9612-bba264f7c727 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:46.850073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e317fbd-7398-4ecd-9612-bba264f7c727 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:46.850073 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e317fbd-7398-4ecd-9612-bba264f7c727 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:46.851425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 660/1320] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:46.856935 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9e3bc-d31e-4415-b096-374292be7582 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:46.857422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9e3bc-d31e-4415-b096-374292be7582 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:46.857523 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9e3bc-d31e-4415-b096-374292be7582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:46.921846 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-9ed9e3bc-d31e-4415-b096-374292be7582 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:46.923081 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-9ed9e3bc-d31e-4415-b096-374292be7582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:47.015812 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 661/1321] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:46:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:47.053463 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e317fbd-7398-4ecd-9612-bba264f7c727 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:46:47.055029 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-3e317fbd-7398-4ecd-9612-bba264f7c727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:47.154718 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e317fbd-7398-4ecd-9612-bba264f7c727 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:46:47.226290 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ed6e6-6a59-4f6d-97e4-8da21d2922cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:46:47.226938 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ed6e6-6a59-4f6d-97e4-8da21d2922cd service neutron] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:47.227228 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ed6e6-6a59-4f6d-97e4-8da21d2922cd service neutron] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:47.227513 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ed6e6-6a59-4f6d-97e4-8da21d2922cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:47.229202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-c56ed6e6-6a59-4f6d-97e4-8da21d2922cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:47.332953 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ed6e6-6a59-4f6d-97e4-8da21d2922cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:46:47.334412 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ed6e6-6a59-4f6d-97e4-8da21d2922cd service neutron] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:46:47.351831 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 661/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:46:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:46:50.253027 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863ba5-dd0d-4003-850e-35da3fbc8cc1 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:50.253580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863ba5-dd0d-4003-850e-35da3fbc8cc1 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:50.253580 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863ba5-dd0d-4003-850e-35da3fbc8cc1 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:50.254968 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 662/1323] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:50.260224 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6970a-9f22-403c-9862-74a46d7974ea None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:50.260689 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6970a-9f22-403c-9862-74a46d7974ea None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:50.260981 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6970a-9f22-403c-9862-74a46d7974ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:50.324141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-01d6970a-9f22-403c-9862-74a46d7974ea None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:50.325388 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-01d6970a-9f22-403c-9862-74a46d7974ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:50.420288 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 662/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:50.456052 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1863ba5-dd0d-4003-850e-35da3fbc8cc1 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:46:50.457729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-b1863ba5-dd0d-4003-850e-35da3fbc8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:50.561479 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1863ba5-dd0d-4003-850e-35da3fbc8cc1 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:46:50.633582 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73585134-569b-49e7-a4d9-2cee495e5edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:46:50.634246 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73585134-569b-49e7-a4d9-2cee495e5edb service neutron] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:50.634541 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73585134-569b-49e7-a4d9-2cee495e5edb service neutron] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:50.634825 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73585134-569b-49e7-a4d9-2cee495e5edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:50.637156 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-73585134-569b-49e7-a4d9-2cee495e5edb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:50.742423 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73585134-569b-49e7-a4d9-2cee495e5edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:46:50.743876 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73585134-569b-49e7-a4d9-2cee495e5edb service neutron] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:46:50.762454 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 663/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:46:53.124330 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f644da-e6d5-4422-acbe-ce785a370aaf None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:53.124864 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f644da-e6d5-4422-acbe-ce785a370aaf None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:53.124864 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f644da-e6d5-4422-acbe-ce785a370aaf None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:53.126420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 663/1326] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:53.131729 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a454a0-5d9b-4e92-92c1-5ef08c8923ed None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:53.132222 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a454a0-5d9b-4e92-92c1-5ef08c8923ed None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:53.132470 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a454a0-5d9b-4e92-92c1-5ef08c8923ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:53.195583 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-29a454a0-5d9b-4e92-92c1-5ef08c8923ed None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:53.196739 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-29a454a0-5d9b-4e92-92c1-5ef08c8923ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:53.293093 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 664/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:53.327524 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f644da-e6d5-4422-acbe-ce785a370aaf None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:46:53.329217 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-89f644da-e6d5-4422-acbe-ce785a370aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:53.430897 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f644da-e6d5-4422-acbe-ce785a370aaf None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:46:53.502357 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6b346-5419-49e2-a8e6-45371013d0f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:46:53.503009 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6b346-5419-49e2-a8e6-45371013d0f0 service neutron] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:53.503295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6b346-5419-49e2-a8e6-45371013d0f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:53.503575 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6b346-5419-49e2-a8e6-45371013d0f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:53.505248 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-daa6b346-5419-49e2-a8e6-45371013d0f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:53.618631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6b346-5419-49e2-a8e6-45371013d0f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:46:53.620601 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6b346-5419-49e2-a8e6-45371013d0f0 service neutron] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:46:53.638664 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 664/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:46:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:46:55.937578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef543316-9c9f-4338-860c-67cbe47bfd59 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:55.938046 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef543316-9c9f-4338-860c-67cbe47bfd59 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:55.938182 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef543316-9c9f-4338-860c-67cbe47bfd59 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:55.939587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 665/1329] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:55.944693 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e87936-b6b8-4fff-877d-be0872a019fb None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:55.945104 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e87936-b6b8-4fff-877d-be0872a019fb None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:55.945295 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e87936-b6b8-4fff-877d-be0872a019fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:56.009093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-71e87936-b6b8-4fff-877d-be0872a019fb None None] MFA Rules not processed for user `2a6b5024341c429e891c0f92594e0c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:56.011219 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-71e87936-b6b8-4fff-877d-be0872a019fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:56.117402 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 665/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:46:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:56.154919 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef543316-9c9f-4338-860c-67cbe47bfd59 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:46:56.156454 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-ef543316-9c9f-4338-860c-67cbe47bfd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:56.257666 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef543316-9c9f-4338-860c-67cbe47bfd59 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:46:56.327765 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71de193-290b-49fc-a4bf-10ce760d95c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:46:56.328422 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71de193-290b-49fc-a4bf-10ce760d95c6 service neutron] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:56.328714 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71de193-290b-49fc-a4bf-10ce760d95c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:56.328997 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71de193-290b-49fc-a4bf-10ce760d95c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:56.330669 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-d71de193-290b-49fc-a4bf-10ce760d95c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:56.445638 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71de193-290b-49fc-a4bf-10ce760d95c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:46:56.447087 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71de193-290b-49fc-a4bf-10ce760d95c6 service neutron] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:46:56.465113 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 666/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:46:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:46:58.918710 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63cacb-846d-432b-9c66-63f6487169e8 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:58.919124 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63cacb-846d-432b-9c66-63f6487169e8 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:58.919275 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63cacb-846d-432b-9c66-63f6487169e8 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:58.920651 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 666/1332] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:46:58.926054 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db756-11c4-417b-952b-27dceb44af86 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:58.926457 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db756-11c4-417b-952b-27dceb44af86 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:58.926631 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db756-11c4-417b-952b-27dceb44af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:58.989177 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-db1db756-11c4-417b-952b-27dceb44af86 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:46:58.990390 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-db1db756-11c4-417b-952b-27dceb44af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:59.082895 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 667/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:46:59.119048 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63cacb-846d-432b-9c66-63f6487169e8 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:46:59.120578 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-ba63cacb-846d-432b-9c66-63f6487169e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:59.221503 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63cacb-846d-432b-9c66-63f6487169e8 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:46:59.290982 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa63120-3d95-4663-a37f-eba867fe76b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:46:59.291629 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa63120-3d95-4663-a37f-eba867fe76b7 service neutron] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:46:59.291917 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa63120-3d95-4663-a37f-eba867fe76b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:46:59.292197 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa63120-3d95-4663-a37f-eba867fe76b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:46:59.293862 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-8fa63120-3d95-4663-a37f-eba867fe76b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:46:59.397633 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa63120-3d95-4663-a37f-eba867fe76b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:46:59.399056 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa63120-3d95-4663-a37f-eba867fe76b7 service neutron] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:46:59.416972 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 667/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 15:47:01.699057 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc718c-6360-4fce-9e2b-72b52407d56a None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:01.699656 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc718c-6360-4fce-9e2b-72b52407d56a None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:01.699656 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc718c-6360-4fce-9e2b-72b52407d56a None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:01.701016 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 668/1335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:01.706337 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00c8d7-d956-4898-bdd6-3d128cbe58c9 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:01.706768 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00c8d7-d956-4898-bdd6-3d128cbe58c9 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:01.706912 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00c8d7-d956-4898-bdd6-3d128cbe58c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:01.769783 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-5c00c8d7-d956-4898-bdd6-3d128cbe58c9 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:01.770975 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-5c00c8d7-d956-4898-bdd6-3d128cbe58c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:01.863887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 668/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:01.903425 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc718c-6360-4fce-9e2b-72b52407d56a None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:01.904977 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-bdfc718c-6360-4fce-9e2b-72b52407d56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:02.005009 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc718c-6360-4fce-9e2b-72b52407d56a None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:02.074027 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a713b-873f-43b8-899f-a5f7e417c8a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:02.074672 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a713b-873f-43b8-899f-a5f7e417c8a4 service neutron] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:02.074958 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a713b-873f-43b8-899f-a5f7e417c8a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:02.075268 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a713b-873f-43b8-899f-a5f7e417c8a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:02.076893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-c19a713b-873f-43b8-899f-a5f7e417c8a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:02.180804 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a713b-873f-43b8-899f-a5f7e417c8a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:02.182782 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a713b-873f-43b8-899f-a5f7e417c8a4 service neutron] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:02.200320 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 669/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:47:04.808058 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d912d0-4586-47a5-be0c-628949418a5a None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:04.808606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d912d0-4586-47a5-be0c-628949418a5a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:04.808606 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d912d0-4586-47a5-be0c-628949418a5a None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:04.809998 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 669/1338] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:04.815308 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f5c0-511e-45d2-a075-3df0ddf59a1d None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:04.815719 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f5c0-511e-45d2-a075-3df0ddf59a1d None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:04.815883 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8f5c0-511e-45d2-a075-3df0ddf59a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:04.878941 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-75a8f5c0-511e-45d2-a075-3df0ddf59a1d None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:04.880159 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-75a8f5c0-511e-45d2-a075-3df0ddf59a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:04.974228 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 670/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:05.009664 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d912d0-4586-47a5-be0c-628949418a5a None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:05.011205 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-50d912d0-4586-47a5-be0c-628949418a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:05.112456 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d912d0-4586-47a5-be0c-628949418a5a None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:05.183271 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480940-eaa2-4ea5-a867-210bad7d655c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:05.183919 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480940-eaa2-4ea5-a867-210bad7d655c service neutron] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:05.184213 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480940-eaa2-4ea5-a867-210bad7d655c service neutron] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:05.184498 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480940-eaa2-4ea5-a867-210bad7d655c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:05.186199 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-c4480940-eaa2-4ea5-a867-210bad7d655c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:05.291909 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4480940-eaa2-4ea5-a867-210bad7d655c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:05.293383 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4480940-eaa2-4ea5-a867-210bad7d655c service neutron] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:05.310666 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 670/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:47:09.886927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293b6d7-75c7-4df7-a85d-814b12a8d66c None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:09.887491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293b6d7-75c7-4df7-a85d-814b12a8d66c None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:09.887491 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293b6d7-75c7-4df7-a85d-814b12a8d66c None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:09.888869 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 671/1341] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:09.894209 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0668b-4b24-44f5-a7b8-e54a62cb3d7e None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:09.894887 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0668b-4b24-44f5-a7b8-e54a62cb3d7e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:09.895185 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0668b-4b24-44f5-a7b8-e54a62cb3d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:09.966006 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-4ee0668b-4b24-44f5-a7b8-e54a62cb3d7e None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:09.967364 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-4ee0668b-4b24-44f5-a7b8-e54a62cb3d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:10.067093 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 671/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:10.105882 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293b6d7-75c7-4df7-a85d-814b12a8d66c None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:10.108135 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-d293b6d7-75c7-4df7-a85d-814b12a8d66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:10.202378 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293b6d7-75c7-4df7-a85d-814b12a8d66c None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:10.206006 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394d356-72b5-42a6-a628-567f791401bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:10.206663 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394d356-72b5-42a6-a628-567f791401bb service neutron] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:10.206952 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394d356-72b5-42a6-a628-567f791401bb service neutron] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:10.207235 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394d356-72b5-42a6-a628-567f791401bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:10.216364 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394d356-72b5-42a6-a628-567f791401bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:10.218036 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394d356-72b5-42a6-a628-567f791401bb service neutron] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:10.227488 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 672/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:47:13.081307 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4358de-bc16-4b16-8890-1ed5b474f91d None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:13.081928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4358de-bc16-4b16-8890-1ed5b474f91d None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:13.081928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4358de-bc16-4b16-8890-1ed5b474f91d None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:13.083257 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 672/1344] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:13.089140 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91315490-511a-44de-91b4-6972c78d764e None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:13.089599 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91315490-511a-44de-91b4-6972c78d764e None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:13.089833 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91315490-511a-44de-91b4-6972c78d764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:13.117527 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-91315490-511a-44de-91b4-6972c78d764e None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:13.118661 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-91315490-511a-44de-91b4-6972c78d764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:13.193700 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 673/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:13.230546 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4358de-bc16-4b16-8890-1ed5b474f91d None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:13.232241 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-6a4358de-bc16-4b16-8890-1ed5b474f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:13.245471 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4358de-bc16-4b16-8890-1ed5b474f91d None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:13.248942 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fa283-e435-4b61-beb5-bfdbe6857cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:13.249611 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fa283-e435-4b61-beb5-bfdbe6857cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:13.249903 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fa283-e435-4b61-beb5-bfdbe6857cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:13.250186 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fa283-e435-4b61-beb5-bfdbe6857cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:13.258881 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fa283-e435-4b61-beb5-bfdbe6857cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:13.260877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7fa283-e435-4b61-beb5-bfdbe6857cf4 service neutron] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:13.270398 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 673/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:47:17.865680 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa35b72-6cec-419d-90a5-035de4ae2157 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:17.866208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa35b72-6cec-419d-90a5-035de4ae2157 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:17.866208 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa35b72-6cec-419d-90a5-035de4ae2157 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:17.867516 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 674/1347] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:17.873232 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00421d53-fcb8-4a43-91bf-cc9bd238c367 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:17.873725 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00421d53-fcb8-4a43-91bf-cc9bd238c367 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:17.873976 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00421d53-fcb8-4a43-91bf-cc9bd238c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:17.892500 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-00421d53-fcb8-4a43-91bf-cc9bd238c367 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:17.893757 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-00421d53-fcb8-4a43-91bf-cc9bd238c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:17.901836 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 674/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:17.939539 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa35b72-6cec-419d-90a5-035de4ae2157 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:17.952106 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa35b72-6cec-419d-90a5-035de4ae2157 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:17.955640 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf505c0-f19f-4ec4-b8ac-234abe99c64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88be715120ee4267a4a206b71fa21224', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:17.956315 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf505c0-f19f-4ec4-b8ac-234abe99c64b service neutron] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:17.956601 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf505c0-f19f-4ec4-b8ac-234abe99c64b service neutron] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:17.956880 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf505c0-f19f-4ec4-b8ac-234abe99c64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:17.965857 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf505c0-f19f-4ec4-b8ac-234abe99c64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:17.967542 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf505c0-f19f-4ec4-b8ac-234abe99c64b service neutron] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:17.976683 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 675/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:47:23.467793 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a745b4-c572-4b39-b24f-cf7738552191 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:23.468651 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a745b4-c572-4b39-b24f-cf7738552191 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:23.469137 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a745b4-c572-4b39-b24f-cf7738552191 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:23.471030 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 675/1350] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:23.480518 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c05188-fb8c-4b80-9e0c-f7d0d163515a None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:23.480960 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c05188-fb8c-4b80-9e0c-f7d0d163515a None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:23.481267 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c05188-fb8c-4b80-9e0c-f7d0d163515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:23.612168 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-29c05188-fb8c-4b80-9e0c-f7d0d163515a None None] MFA Rules not processed for user `03ad505fa13b43ea85f25ec2e8bd3cd3`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:23.613406 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-29c05188-fb8c-4b80-9e0c-f7d0d163515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:23.693834 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 676/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:47:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:28.340662 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b72ea-f686-4027-a890-2c3cf7c98688 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:28.341264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b72ea-f686-4027-a890-2c3cf7c98688 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:28.341264 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b72ea-f686-4027-a890-2c3cf7c98688 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:28.343005 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 676/1352] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:28.350789 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9dca-ecd6-4754-a494-6f5e680a4d92 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:28.351244 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9dca-ecd6-4754-a494-6f5e680a4d92 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:28.351373 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9dca-ecd6-4754-a494-6f5e680a4d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:28.371166 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-8e4f9dca-ecd6-4754-a494-6f5e680a4d92 None None] MFA Rules not processed for user `03ad505fa13b43ea85f25ec2e8bd3cd3`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:28.372459 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-8e4f9dca-ecd6-4754-a494-6f5e680a4d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:28.381069 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 677/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:47:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:29.927303 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06720f1e-45b5-479e-a2bd-4bd2d0f91a8e None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:29.927809 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06720f1e-45b5-479e-a2bd-4bd2d0f91a8e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:29.927914 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06720f1e-45b5-479e-a2bd-4bd2d0f91a8e None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:29.929272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 677/1354] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 9 15:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 09 15:47:29.934727 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42ca9c-b7eb-4783-a586-431c06096cc2 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:29.935153 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42ca9c-b7eb-4783-a586-431c06096cc2 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:29.935283 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42ca9c-b7eb-4783-a586-431c06096cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:29.953496 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-fb42ca9c-b7eb-4783-a586-431c06096cc2 None None] MFA Rules not processed for user `03ad505fa13b43ea85f25ec2e8bd3cd3`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:29.954693 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-fb42ca9c-b7eb-4783-a586-431c06096cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:29.963419 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 678/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 9 15:47:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:33.841622 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc77ce-5422-4743-a8a3-59fc74aceae9 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:33.842138 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc77ce-5422-4743-a8a3-59fc74aceae9 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:33.842236 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc77ce-5422-4743-a8a3-59fc74aceae9 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:33.845294 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 678/1356] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:33.849392 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eade7f-51bf-4541-873b-c93ce684b293 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:33.851504 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eade7f-51bf-4541-873b-c93ce684b293 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:33.852074 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eade7f-51bf-4541-873b-c93ce684b293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:33.871865 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-04eade7f-51bf-4541-873b-c93ce684b293 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:33.873111 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-04eade7f-51bf-4541-873b-c93ce684b293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:33.881406 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 679/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:33.930272 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860635a8-82dc-4225-8574-1a658a2dafc1 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:33.930569 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860635a8-82dc-4225-8574-1a658a2dafc1 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:33.931148 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860635a8-82dc-4225-8574-1a658a2dafc1 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:33.932521 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 679/1358] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 9 15:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:33.938039 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c0c5b-34f8-48cf-a01b-f872b6fad23a None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:33.938412 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c0c5b-34f8-48cf-a01b-f872b6fad23a None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:33.938795 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c0c5b-34f8-48cf-a01b-f872b6fad23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:33.995215 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-857c0c5b-34f8-48cf-a01b-f872b6fad23a None None] MFA Rules not processed for user `ef8546c698f9455cb81264bcee0d2344`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:33.997114 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-857c0c5b-34f8-48cf-a01b-f872b6fad23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:34.079680 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 680/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 9 15:47:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:34.088171 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860635a8-82dc-4225-8574-1a658a2dafc1 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:34.103223 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860635a8-82dc-4225-8574-1a658a2dafc1 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:34.108785 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d119743-f070-4d1a-9bca-08dc5b4bd211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8546c698f9455cb81264bcee0d2344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:34.109575 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119743-f070-4d1a-9bca-08dc5b4bd211 service nova] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:34.109990 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119743-f070-4d1a-9bca-08dc5b4bd211 service nova] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:34.110420 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119743-f070-4d1a-9bca-08dc5b4bd211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:34.119698 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d119743-f070-4d1a-9bca-08dc5b4bd211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:34.121455 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d119743-f070-4d1a-9bca-08dc5b4bd211 service nova] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:34.130794 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 680/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:47:36.597581 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a655c-0dea-49e3-a99b-8589951c4101 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:36.598160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a655c-0dea-49e3-a99b-8589951c4101 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:36.598160 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a655c-0dea-49e3-a99b-8589951c4101 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:36.599559 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 681/1361] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:36.604946 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b49242-4008-4ac6-93a7-221790670c95 None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:36.605480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b49242-4008-4ac6-93a7-221790670c95 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:36.605583 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b49242-4008-4ac6-93a7-221790670c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:36.624644 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-55b49242-4008-4ac6-93a7-221790670c95 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:36.625928 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-55b49242-4008-4ac6-93a7-221790670c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:36.634836 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 681/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:36.679126 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a655c-0dea-49e3-a99b-8589951c4101 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:36.694056 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a655c-0dea-49e3-a99b-8589951c4101 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:36.700252 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd499d7a-dc4e-482a-8ac2-e55d1d6aceb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8546c698f9455cb81264bcee0d2344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:36.700899 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499d7a-dc4e-482a-8ac2-e55d1d6aceb2 service nova] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:36.701232 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499d7a-dc4e-482a-8ac2-e55d1d6aceb2 service nova] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:36.701519 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd499d7a-dc4e-482a-8ac2-e55d1d6aceb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:36.709972 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd499d7a-dc4e-482a-8ac2-e55d1d6aceb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:36.711581 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd499d7a-dc4e-482a-8ac2-e55d1d6aceb2 service nova] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:36.721773 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 682/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:47:39.265877 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23545eb2-b751-4af4-9a22-65d64fa0b88a None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:39.266425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23545eb2-b751-4af4-9a22-65d64fa0b88a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:39.266425 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23545eb2-b751-4af4-9a22-65d64fa0b88a None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:39.267703 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 682/1364] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:39.273158 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a2b1b-5d94-425f-9544-1b9cd7f10e70 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:39.273569 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a2b1b-5d94-425f-9544-1b9cd7f10e70 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:39.273755 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a2b1b-5d94-425f-9544-1b9cd7f10e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:39.292007 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-a39a2b1b-5d94-425f-9544-1b9cd7f10e70 None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:39.293334 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-a39a2b1b-5d94-425f-9544-1b9cd7f10e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:39.301440 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 683/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:47:39.349836 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce26a7-af55-4332-8ad7-dabc04b8d6f2 None None] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:39.350125 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce26a7-af55-4332-8ad7-dabc04b8d6f2 None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:39.350594 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce26a7-af55-4332-8ad7-dabc04b8d6f2 None None] PATH_INFO: `/` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:39.352993 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 683/1366] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 9 15:47:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:39.359437 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fc400-c4bb-49b7-8015-e1a814d045b6 None None] REQUEST_METHOD: `POST` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:39.359964 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fc400-c4bb-49b7-8015-e1a814d045b6 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:39.360410 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fc400-c4bb-49b7-8015-e1a814d045b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:39.381587 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.auth.core [None req-bc0fc400-c4bb-49b7-8015-e1a814d045b6 None None] MFA Rules not processed for user `ef8546c698f9455cb81264bcee0d2344`. Rule list: `[]` (Enabled: `True`). {{(pid=275711) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:39.383611 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.fernet_utils [None req-bc0fc400-c4bb-49b7-8015-e1a814d045b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275711) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:39.396014 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 684/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 9 15:47:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 09 15:47:39.403632 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce26a7-af55-4332-8ad7-dabc04b8d6f2 None None] Authenticating user token {{(pid=275712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:39.418259 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce26a7-af55-4332-8ad7-dabc04b8d6f2 None None] Validating token access rules against request {{(pid=275712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:39.423944 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f8298-3fd1-46ed-8b5f-acbec639c42c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8546c698f9455cb81264bcee0d2344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:39.424698 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8298-3fd1-46ed-8b5f-acbec639c42c service nova] REQUEST_METHOD: `GET` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:39.425085 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8298-3fd1-46ed-8b5f-acbec639c42c service nova] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:39.425447 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8298-3fd1-46ed-8b5f-acbec639c42c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:39.434631 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8298-3fd1-46ed-8b5f-acbec639c42c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:39.436363 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f8298-3fd1-46ed-8b5f-acbec639c42c service nova] RBAC: Authorization granted {{(pid=275712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:39.448081 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 684/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 09 15:47:40.563912 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7b8cf-eaf4-4345-864b-4ee07b8618ee None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:40.564451 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7b8cf-eaf4-4345-864b-4ee07b8618ee None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:40.564451 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7b8cf-eaf4-4345-864b-4ee07b8618ee None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:40.565896 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 685/1369] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:40.571202 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21d90c-95b9-4f6a-bd69-6971709bf26e None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:40.572141 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21d90c-95b9-4f6a-bd69-6971709bf26e None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:40.572290 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21d90c-95b9-4f6a-bd69-6971709bf26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:40.590988 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-2c21d90c-95b9-4f6a-bd69-6971709bf26e None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:40.592688 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-2c21d90c-95b9-4f6a-bd69-6971709bf26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:40.609097 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 685/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:41.790331 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97e15-71fe-485c-b69d-4db2995079a0 None None] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:41.790927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97e15-71fe-485c-b69d-4db2995079a0 None None] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:41.790927 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c97e15-71fe-485c-b69d-4db2995079a0 None None] PATH_INFO: `/` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:41.792289 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 686/1371] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 9 15:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 09 15:47:41.797561 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829132ed-7da7-44b5-9df6-870b29cfca6a None None] REQUEST_METHOD: `POST` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:41.798086 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829132ed-7da7-44b5-9df6-870b29cfca6a None None] SCRIPT_NAME: `/identity` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:41.798427 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829132ed-7da7-44b5-9df6-870b29cfca6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=275712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:41.816480 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.auth.core [None req-829132ed-7da7-44b5-9df6-870b29cfca6a None None] MFA Rules not processed for user `d2a1baaf61604444b2c20a19956c1591`. Rule list: `[]` (Enabled: `True`). {{(pid=275712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 09 15:47:41.817714 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: DEBUG keystone.common.fernet_utils [None req-829132ed-7da7-44b5-9df6-870b29cfca6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=275712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 09 15:47:41.827008 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: [pid: 275712|app: 0|req: 686/1372] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 9 15:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 09 15:47:41.871993 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c97e15-71fe-485c-b69d-4db2995079a0 None None] Authenticating user token {{(pid=275711) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 09 15:47:41.888102 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c97e15-71fe-485c-b69d-4db2995079a0 None None] Validating token access rules against request {{(pid=275711) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 09 15:47:41.893578 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0e894-e221-4aa6-90c6-33992cc6c73a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8546c698f9455cb81264bcee0d2344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '290042e8fb734d94baf915970d7d7b95', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=275711) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 09 15:47:41.894233 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0e894-e221-4aa6-90c6-33992cc6c73a service nova] REQUEST_METHOD: `GET` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 09 15:47:41.894528 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0e894-e221-4aa6-90c6-33992cc6c73a service nova] SCRIPT_NAME: `/identity` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 09 15:47:41.894817 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0e894-e221-4aa6-90c6-33992cc6c73a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=275711) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 09 15:47:41.903770 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0e894-e221-4aa6-90c6-33992cc6c73a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 09 15:47:41.905437 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0e894-e221-4aa6-90c6-33992cc6c73a service nova] RBAC: Authorization granted {{(pid=275711) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 09 15:47:41.915893 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: [pid: 275711|app: 0|req: 687/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 9 15:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 09 15:48:09.078516 dv-cinder-840678-3-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 09 15:48:09.079164 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: Mon May 9 15:48:09 2022 - graceful shutdown triggered... May 09 15:48:09.079806 dv-cinder-840678-3-iscsi devstack@keystone.service[275711]: Gracefully killing worker 1 (pid: 275711)... May 09 15:48:09.080250 dv-cinder-840678-3-iscsi devstack@keystone.service[275712]: Gracefully killing worker 2 (pid: 275712)... May 09 15:48:10.080119 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: worker 1 buried after 1 seconds May 09 15:48:10.080119 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: worker 2 buried after 1 seconds May 09 15:48:10.080119 dv-cinder-840678-3-iscsi devstack@keystone.service[275699]: goodbye to uWSGI. May 09 15:48:10.081773 dv-cinder-840678-3-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 09 15:48:10.082322 dv-cinder-840678-3-iscsi systemd[1]: Stopped Devstack devstack@keystone.service.